Correcting what cannot be corrected: rethinking publication bias analysis methods in clinical meta-analyses

Area of Science:

• Biostatistics
• Clinical Epidemiology
• Medical Research Synthesis

Background:

• Publication bias significantly distorts evidence in meta-analyses, particularly in clinical research.
• Existing statistical adjustments for publication bias have limitations in recovering selectively generated or disseminated data.
• Complexities in clinical evidence syntheses, including heterogeneity and varied dissemination pathways, exacerbate publication bias challenges.

Purpose of the Study:

• To reframe publication bias assessment and adjustment methods as inferential processes, not definitive corrections.
• To critically review classical detection and modern adjustment methods for publication bias, highlighting assumptions and failure modes.
• To provide a framework for evaluating these methods and address emerging challenges in evidence synthesis.

Main Methods:

• Review of classical publication bias detection methods: funnel plots, small-study effect tests, P-value approaches.
• Examination of adjustment methods: trim-and-fill, selection models, regression-based approaches, Bayesian methods.
• Illustrative worked example demonstrating divergent results from different adjustment methods on the same data.

Main Results:

• Detection methods serve as stress tests for model adequacy, not definitive bias detectors.
• Adjustment methods are assumption-dependent, often yielding conflicting results when applied to the same evidence base.
• Common misuses and emerging challenges (preprints, AI) complicate the interpretation of publication bias analyses.

Conclusions:

• Publication bias adjustment methods should be viewed as inferential tools with inherent limitations.
• The strength of clinical conclusions must align with the robustness of the underlying data and the chosen bias assessment methods.
• A critical, assumption-aware approach is necessary for authors, reviewers, and editors when interpreting evidence syntheses.