Related Experiment Video
Updated: Aug 20, 2025

Modeling the Size Spectrum for Macroinvertebrates and Fishes in Stream Ecosystems
Published on: July 30, 2019
How much does the typical ecological meta-analysis overestimate the true mean effect size?
1Department of Biological Sciences University of Calgary Calgary Alberta Canada.
Abstract:
Many primary research studies in ecology are underpowered, providing very imprecise estimates of effect size. Meta-analyses partially mitigate this imprecision by combining data from different studies. But meta-analytic estimates of mean effect size may still remain imprecise, particularly if the meta-analysis includes a small number of studies. Imprecise, large-magnitude estimates of mean effect size from small meta-analyses likely would shrink if additional studies were conducted (regression towards the mean). Here, I propose a way to estimate and correct this regression to the mean, using meta-meta-analysis (meta-analysis of meta-analyses). Hierarchical random effects meta-meta-analysis shrinks estimated mean effect sizes from different meta-analyses towards the grand mean, bringing those estimated means closer on average to their unknown true values. The intuition is that, if a meta-analysis reports a mean effect size much larger in magnitude than that reported by other meta-analyses, that large mean effect size likely is an overestimate. This intuition holds even if different meta-analyses of different topics have different true mean effect sizes. Drawing on a compilation of data from hundreds of ecological meta-analyses, I find that the typical (median) ecological meta-analysis overestimates the absolute magnitude of the true mean effect size by ~10%. Some small ecological meta-analyses overestimate the magnitude of the true mean effect size by >50%. Meta-meta-analysis is a promising tool for improving the accuracy of meta-analytic estimates of mean effect size, particularly estimates based on just a few studies.
More Related Videos
08:36Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
Published on: April 19, 2024
06:26Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
Published on: November 27, 2019
Related Concept Videos
Regression Toward the Mean
Bias in Epidemiological Studies
One-Way ANOVA: Equal Sample Sizes
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Testing a Claim about Mean: Known Population SD
Estimating a population mean requires the samples to be distributed normally. The data should be collected from the randomly selected samples having no sampling bias. The sample size needed to be higher than 30, and most importantly, the population standard deviation should be already known.
In most realistic situations, the population standard deviation is often unknown, but in rare circumstances, when it...
What are Estimates?
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...
Confounding in Epidemiological Studies