Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis

Area of Science:

• Psychology
• Cognitive Science
• Psychometrics

Background:

• Intelligence research is a vast field with numerous studies examining cognitive abilities.
• Assessing the overall landscape of effect sizes, statistical power, and potential biases is crucial for scientific rigor.

Purpose of the Study:

• To meta-analytically estimate the average effect size, median statistical power, and evidence for bias in intelligence research.
• To investigate variations in these metrics across different types of intelligence studies.
• To evaluate the prevalence of small-study effects and other potential biases.

Main Methods:

• Analysis of 2442 effect sizes from 131 meta-analyses published between 1984 and 2014.
• Calculation of average effect size (Pearson's correlation), median sample size, and median statistical power.
• Comparison of metrics across study types: correlational, group differences, experimental, toxicology, and behavior genetics.

Main Results:

• The average effect size was a Pearson's correlation of 0.26, with a median sample size of 60.
• Median power to detect small, medium, and large effects was 11.9%, 54.5%, and 93.9%, respectively.
• Evidence for small-study effects was found, suggesting potential publication bias and overestimated effects, though not differing by study type.

Conclusions:

• Intelligence research exhibits signs of low statistical power and publication bias, though potentially less severe than in other fields.
• Variations in effect size and power exist across different intelligence study methodologies.
• No convincing evidence for decline effect, US effect, or citation bias was observed across the analyzed meta-analyses.