Correcting bias in the meta-analysis of correlations

Area of Science:

• Psychometrics
• Statistical Methodology
• Meta-Analysis

Background:

• Conventional meta-analyses of correlation coefficients are susceptible to bias.
• Standard errors of correlation coefficients are dependent on the coefficient's magnitude, introducing bias in inverse-variance weighted averages.
• Existing biases persist even under ideal conditions, excluding publication bias or p-hacking.

Purpose of the Study:

• To demonstrate the inherent bias in conventional meta-analyses of correlation coefficients.
• To explain the underlying reasons for this bias.
• To propose and evaluate solutions for mitigating these biases.

Main Methods:

• Analysis of bias in inverse-variance weighted averages of correlation coefficients.
• Evaluation of Fisher's z-transformation as a bias reduction technique.
• Development and application of a novel small-sample adjustment for Fisher's z-transformation.

Main Results:

• All conventional meta-analyses of correlation coefficients exhibit bias.
• Fisher's z-transformation significantly reduces but does not entirely eliminate bias.
• The proposed small-sample adjustment renders remaining bias scientifically trivial, especially for typical psychology sample sizes (n < 200).

Conclusions:

• Standard meta-analysis techniques for correlation coefficients are fundamentally biased.
• Fisher's z-transformation is an improvement but insufficient on its own.
• The novel small-sample adjustment offers a robust solution for accurate meta-analysis of correlation coefficients in psychology and other fields with small sample sizes.