Related Experiment Videos
Effect Size Calculation Procedures in Exercise Intervention Meta-Analyses: A Scoping Review
Simon Nolte1, Theresa Siepe2,3, Jennifer Murphy4
1Institute of Exercise Training and Sport Informatics, German Sport University, Cologne, Germany. s.nolte@dshs-koeln.de.
Background:
Meta-analyses require the calculation of effect sizes for all included studies. The most common experimental design in exercise intervention studies is the pre-post control design. A number of different approaches exist for calculating effect sizes for this design, and it is currently unknown which calculation methods are actually used in the published exercise science literature and how this might impact results.
Methods:
We extracted all meta-analyses (N = 101) published in six journals over a 2-year period where the primary studies used a pre-post control design. We recalculated randomly selected primary study effect sizes in each meta-analysis to determine the calculation methods used and compare them to the reported methods. An example case was used to illustrate the impact of using different calculation methods on meta-analysis outcomes.
Results:
We identified a total of 50 unique combinations of calculation methods among the n = 91 meta-analyses for which we could determine the calculation procedure. Most researchers used change score-based standardized mean differences. These required the imputation of a pre-post correlation, for which most authors used fixed benchmarks instead of evidence-based or previously reported values. Reporting was often incomplete and we identified major issues in the calculation of at least 27 articles. We demonstrate that different calculation methods can yield different meta-analysis outcomes.
Conclusions:
Effect size calculation procedures vary widely, which can affect meta-analysis outcomes and mislead their interpretation. A considerable number of published meta-analytical findings in exercise science may differ when other reasonable or corrected calculation methods are used. We provide recommendations on how to choose an appropriate effect size method for the pre-post control design. We encourage researchers to transparently report and justify the procedures used and perform sensitivity analyses when performing meta-analyses.