Treatment Effects in Randomized and Nonrandomized Studies of Pharmacological Interventions: A Meta-Analysis

Area of Science:

• Pharmacology and Clinical Research Methodology
• Evidence Synthesis and Meta-Analysis
• Drug Efficacy and Safety Assessment

Background:

• Randomized clinical trials (RCTs) are the gold standard for evaluating health interventions.
• Increasing interest exists in utilizing nonrandomized studies for drug efficacy and safety assessments.
• A critical need to compare treatment effect estimates from both study designs is apparent.

Purpose of the Study:

• To compare treatment effect estimates for pharmacological interventions derived from nonrandomized studies versus randomized studies.
• To quantify discrepancies in efficacy and safety conclusions between the two study types.

Main Methods:

• A meta-epidemiological framework was employed, analyzing meta-analyses published between 2009-2018.
• Separate summary effect sizes were calculated for nonrandomized and randomized studies within each meta-analysis.
• Comparison of effect size estimates and assessment of disagreement regarding direction and magnitude of effect.

Main Results:

• Statistical conclusions differed for 37.6% of meta-analyses when considering only one study type.
• No strong evidence of consistent overall differences in treatment effects between study types (ROR 0.95).
• Nonrandomized studies showed increased heterogeneity and, on average, a larger treatment effect (ROR 0.81).

Conclusions:

• While no average difference in effect size exists, nonrandomized studies introduce uncertainty and can misrepresent treatment effects.
• Findings suggest caution when using nonrandomized studies as direct substitutes for RCTs in drug evaluation.
• The reliability of evidence synthesis for new drug therapies may be impacted by the inclusion of nonrandomized studies.