Adapting methods for correcting selective reporting bias in meta-analysis of dependent effect sizes.

Area of Science:

• Meta-analysis and statistical methodology
• Biostatistics and research methodology

Background:

• Selective reporting bias occurs when study results are chosen for publication based on statistical significance or magnitude.
• This bias can lead to overestimated effect sizes, increased Type I error rates, and flawed conclusions in meta-analyses.
• Existing statistical methods for correcting selective reporting bias often assume independence of effect sizes, a condition frequently violated in practice.

Purpose of the Study:

• To evaluate the performance of current selective reporting adjustment methods when effect sizes are dependent.
• To propose and evaluate novel adaptations of adjustment methods that account for dependencies among effect sizes.
• To provide guidance on correcting selective reporting bias in meta-analyses with dependent effect sizes.

Main Methods:

• Conducted a simulation study using dependent effect size estimates.
• Investigated the performance of existing and newly adapted selective reporting adjustment methods.
• Focused on estimating overall average effects, assessing bias, root-mean-squared error, and confidence interval properties.

Main Results:

• Current adjustment methods may perform inadequately when effect sizes are dependent.
• Novel adaptations demonstrated improved performance in correcting selective reporting bias under dependency.
• The proposed multivariate working model and weighting scheme effectively handle effect size dependencies.

Conclusions:

• Selective reporting bias adjustment requires methods that explicitly account for dependencies among effect sizes.
• The novel adaptations offer a more robust approach for meta-analyses with correlated effect sizes.
• Findings provide practical recommendations for meta-analysts to mitigate bias and improve the reliability of systematic reviews.