Assessing meta-regression methods for examining moderator relationships with dependent effect sizes: A Monte Carlo...

Area of Science:

• Biostatistics
• Meta-analysis
• Quantitative Synthesis

Background:

• Dependent effect sizes are common in meta-analysis, complicating standard analytical approaches.
• Accurate statistical inference requires methods that properly account for effect size dependency.

Purpose of the Study:

• To compare the performance of robust variance estimation (RVE) and 3-level modeling against standard methods for meta-regression with dependent effect sizes.
• To evaluate small-sample adjustments for RVE and hypothesis testing methods for 3-level models.

Main Methods:

• Monte Carlo simulation was used to compare statistical methods.
• Evaluated bias in slope estimates, confidence interval width, Type I error rates, and statistical power.
• Compared methods under varying conditions, including moderator placement and study characteristics.

Main Results:

• Standard methods ignoring dependency inflated Type I error rates for moderator significance testing.
• Robust variance estimation (RVE) provided the best Type I error control but resulted in wider confidence intervals and lower statistical power, especially with jackknife adjustments.
• Three-level models demonstrated strong performance, particularly with the likelihood ratio test, offering narrower confidence intervals and higher power than RVE in moderate to large studies.

Conclusions:

• Robust variance estimation and 3-level modeling are superior to standard methods when dealing with dependent effect sizes in meta-regression.
• Three-level models are a promising alternative, especially for meta-analyses with a moderate to large number of studies.
• Method performance is enhanced with moderators at the effect size level, larger study counts, and smaller between-studies variance.