Evaluation of various estimators for standardized mean difference in meta-analysis

Area of Science:

• Biostatistics
• Medical Research Methodology
• Statistical Analysis

Background:

• Meta-analyses commonly use standardized mean differences (SMDs) to synthesize treatment effects.
• Cohen's d and Hedges' g are prevalent estimators for SMDs, but are known to have limitations, particularly with small sample sizes.
• Inconsistencies in current literature and software regarding SMD synthesis methods pose challenges for reproducibility.

Purpose of the Study:

• To comprehensively review and evaluate existing methods for synthesizing standardized mean differences (SMDs) in meta-analyses.
• To compare the performance of different SMD synthesis methods using simulation studies and real-world data.
• To identify the most accurate and unbiased methods for meta-estimation of treatment effects.

Main Methods:

• Extensive simulation studies were conducted to compare the performance of various SMD synthesis methods.
• Analyses of actual datasets were performed to validate findings from simulations.
• Evaluation of conventional methods, including Cohen's d and Hedges' g, and alternative approaches.

Main Results:

• The usual version of Hedges' g can lead to more biased meta-estimation than Cohen's d due to the intrinsic association between point estimates and standard errors.
• Conventional methods for estimating the variance of SMDs are often associated with the point estimate, compromising unbiasedness in meta-analyses.
• Hedges' g's bias-correction factor does not guarantee unbiasedness in the meta-analytic context.

Conclusions:

• Average-adjusted variance estimators are recommended for obtaining unbiased meta-estimates of treatment effects.
• The Hartung-Knapp-Sidik-Jonkman method is advised for accurate confidence interval estimation in meta-analyses.
• Addressing the inconsistencies in SMD synthesis is crucial for improving the reliability and reproducibility of meta-analytic findings.