Quantifying the robustness of primary analysis results: A case study on missing outcome data in pairwise and network...

Area of Science:

• Biostatistics
• Evidence Synthesis
• Health Research Methodology

Background:

• Sensitivity analyses are crucial for evaluating the reliability of systematic review findings.
• Current methods for sensitivity analyses in meta-analysis often lack objective decision rules, relying on subjective comparisons or statistical significance.
• Assumptions about missing data mechanisms can significantly impact meta-analysis results.

Purpose of the Study:

• To propose an objective framework for assessing the robustness of primary analysis results in systematic reviews.
• To develop an intuitive index and a graphical tool to aid in the interpretation of sensitivity analyses.
• To provide clear decision rules for determining the robustness of meta-analysis findings, especially concerning missing outcome data.

Main Methods:

• Development of a novel index utilizing the distribution of estimated treatment effects from primary and alternative re-analyses.
• Comparison of the proposed index against an objective threshold to infer robustness.
• Creation of a graph to visualize and understand the impact of different missing data scenarios on treatment effect estimates.

Main Results:

• The proposed index offers an objective measure to determine the robustness of meta-analysis results.
• The accompanying graph effectively illustrates how different missing data assumptions influence findings.
• The framework provides a clear decision-making process for interpreting sensitivity analyses.

Conclusions:

• The novel index and graphical tool offer an objective and intuitive approach to sensitivity analyses in meta-analysis.
• This framework enhances the reliability and interpretability of systematic review findings, particularly in the presence of missing data.
• The proposed decision framework is directly applicable to pairwise meta-analysis and network meta-analysis, improving the rigor of evidence synthesis.