How much does the typical ecological meta-analysis overestimate the true mean effect size?

Area of Science:

• Ecology
• Statistical Methods

Background:

• Primary ecological studies frequently lack statistical power, leading to imprecise effect size estimates.
• Meta-analyses combine studies but can still produce imprecise mean effect sizes, particularly with few contributing studies.

Purpose of the Study:

• To propose and evaluate a method for estimating and correcting regression to the mean in ecological meta-analyses.
• To improve the accuracy of mean effect size estimates in ecological research.

Main Methods:

• Utilized meta-meta-analysis (a meta-analysis of meta-analyses) with a hierarchical random effects model.
• Applied the method to a compilation of hundreds of ecological meta-analyses.

Main Results:

• The typical ecological meta-analysis overestimates the true mean effect size magnitude by approximately 10%.
• Small meta-analyses can overestimate true effect size magnitude by over 50% due to regression to the mean.

Conclusions:

• Meta-meta-analysis effectively shrinks estimated mean effect sizes towards a grand mean, enhancing accuracy.
• This approach is particularly valuable for refining estimates from meta-analyses with a limited number of studies.