Estimating outcome-specific effects in meta-analyses of multiple outcomes: A simulation study.

Area of Science:

• Statistics
• Psychometrics
• Biostatistics

Background:

• Meta-analysis often involves dependent effect sizes within primary studies.
• Existing methods like multivariate models, three-level models, and robust variance estimation (RVE) primarily focus on overall effect size estimation.
• Estimating outcome-specific effects in the presence of multiple, dependent effect sizes requires robust analytical approaches.

Purpose of the Study:

• To extend previous simulation studies by comparing statistical models for meta-analysis with multiple dependent effect sizes.
• To evaluate the performance of various three-level models and the multivariate model in estimating outcome-specific effects.
• To assess the impact of robust variance (RV) correction on standard error and confidence interval accuracy.

Main Methods:

• Simulation study comparing the performance of different meta-analytic models.
• Inclusion of multivariate models, various three-level model specifications (common vs. outcome-specific effects), and separate three-level models per outcome type.
• Evaluation of a posteriori robust variance (RV) correction on standard error and confidence interval estimation.

Main Results:

• Separate three-level models for each outcome type demonstrated consistent, adequate standard error estimation.
• A posteriori application of RV correction yielded accurate 95% confidence intervals across all tested models, including misspecified ones.
• The robust variance correction ensured adequate Type I error rates when implemented.

Conclusions:

• For meta-analyses with multiple outcome-specific effect sizes, employing separate three-level models is recommended for accurate standard error estimation.
• Robust variance correction is a valuable tool for improving the accuracy of confidence intervals and maintaining appropriate Type I error rates, even with model misspecification.
• This research provides critical insights for selecting appropriate statistical methods in complex meta-analytic scenarios involving dependent effect sizes.