Examining and Interpreting Doi Plot Asymmetry in Meta-Analyses of Randomized Controlled Trials

Area of Science:

• Biostatistics
• Evidence Synthesis
• Research Methodology

Background:

• Systematic reviews and meta-analyses are high-level evidence but vulnerable to publication bias.
• Traditional methods like funnel plots and Egger tests have limitations, including subjectivity and dependence on the number of studies (k).

Purpose of the Study:

• To evaluate the performance of the Doi plot and LFK index for detecting publication bias.
• To compare these novel methods against traditional p-value-based tests using simulation studies.
• To address recent criticisms and assess the robustness of the Doi plot and LFK index.

Main Methods:

• A simulation study was conducted with varying numbers of studies (k=5, 10, 20, 50), sample sizes, and bias levels (ρ=0, -0.3, -0.5, -0.9) using the Copas selection model.
• Diagnostic performance metrics, including sensitivity and specificity, were estimated for the LFK index and Egger test.
• The Doi plot and LFK index were assessed for their visualization and quantification capabilities.

Main Results:

• The LFK index demonstrated consistently higher sensitivity across different k values and bias levels.
• The Egger test's sensitivity was highly dependent on k, decreasing significantly in smaller meta-analyses (k < 20).
• The LFK index showed adjusted specificity with random error, outperforming the Egger test's fixed specificity.

Conclusions:

• The Doi plot and LFK index effectively overcome limitations of traditional methods for publication bias assessment.
• These methods offer robust, k-independent performance, leading to more reliable detection of publication bias.
• A transition to the Doi plot and LFK index is recommended for improved meta-analysis quality.