Bias, coverage, and asymptotic behaviour of random effects meta-analysis: a clinically driven simulation study.

Area of Science:

• Biostatistics
• Clinical Epidemiology

Background:

• Foundational features of meta-analysis are crucial for reliable evidence synthesis.
• The DerSimonian and Laird (D&L) random effects model is widely used but its performance characteristics require further investigation.

Purpose of the Study:

• To evaluate the bias, coverage, and asymptotic behavior of the D&L meta-analysis method using simulations.
• To assess performance across varying trial numbers, sizes, risk levels, and treatment effect extents.

Main Methods:

• Simulated data from randomized controlled trials were used to model risk of untoward events.
• Treatment effect was quantified as relative risk reduction, with effect size estimated by odds ratio.
• Performance metrics included bias, standardized bias, and coverage, compared against prespecified thresholds.

Main Results:

• Bias, standardized bias, and coverage varied significantly with trial characteristics and risk distributions.
• Increasing trial size and number improved performance, but satisfactory results were not consistently achieved.
• Performance was poorer with normal risk distributions compared to constant or narrow uniform distributions. Asymptotic behavior did not demonstrate bias approaching zero.

Conclusions:

• The D&L random effects meta-analysis method demonstrated modest performance at best.
• Asymptotic normality could not be demonstrated, raising questions about the method's validity.
• Findings suggest caution against generic use of the D&L method, warranting replication and extension.