A re-analysis of about 60,000 sparse data meta-analyses suggests that using an adequate method for pooling matters

Area of Science:

• Biostatistics
• Medical Research Methodology
• Evidence Synthesis

Background:

• Sparse data meta-analyses, common in systematic reviews, pose challenges for conventional statistical methods, potentially leading to distorted results.
• One-stage statistical models offer improved performance but are underutilized in practice for meta-analyses with few trials or zero events.

Purpose of the Study:

• To compare the performance of conventional meta-analysis methods against one-stage models using real-world data from the Cochrane Database of Systematic Reviews.
• To evaluate the impact of these methods on statistical precision and confidence interval estimation in scenarios with sparse data (zero or few events).

Main Methods:

• Re-analysis of meta-analyses from the Cochrane Database of Systematic Reviews focusing on zero event and few trial scenarios.
• Application and comparison of one-stage models (Generalised linear mixed model [GLMM], Beta-binomial model [BBM], Bayesian binomial-normal hierarchical model [BNHM-WIP]) against conventional methods (Peto-Odds-ratio [PETO], DerSimonian-Laird [DL], Paule-Mandel [PM], Restricted maximum likelihood [REML]).

Main Results:

• While treatment effect estimates were similar across methods, substantial variability in statistical precision was observed.
• Conventional methods produced narrower confidence intervals in zero event scenarios, whereas the Beta-binomial model yielded the widest intervals in few trial scenarios.
• Significance outcomes frequently differed between one-stage models and conventional methods in the few trials scenario.

Conclusions:

• One-stage models are recommended for meta-analyses with zero event trials, aligning with existing simulations and guidelines.
• For few trial scenarios, the Beta-binomial model combined with Paule-Mandel or REML for sensitivity analysis may provide conservative results.
• Careful selection of meta-analysis methodology is crucial for reliable evidence synthesis, particularly with sparse data.