Nonrandomized studies using causal-modeling may give different answers than RCTs: a meta-epidemiological study

Area of Science:

• Epidemiology
• Biostatistics
• Health Services Research

Background:

• Randomized controlled trials (RCTs) are the gold standard for evaluating treatment effects.
• Nonrandomized studies using causal modeling, such as marginal structural models (MSM-studies), offer an alternative using real-world data.
• Evaluating the agreement between these study designs is crucial for evidence-based healthcare.

Purpose of the Study:

• To compare the estimated treatment effects between MSM-studies and RCTs.
• To assess the concordance in direction, effect size, and confidence intervals of treatment effects.
• To identify potential systematic disagreements between the two study designs.

Main Methods:

• A meta-epidemiological study design was employed.
• MSM-studies with effect estimates on healthcare outcomes were systematically identified.
• RCTs addressing the same clinical questions were sought for comparison.

Main Results:

• The analysis included 19 MSM-studies and 141 RCTs.
• MSM-studies showed effect estimates in the opposite direction to RCTs for 42% of questions.
• Effect estimates between study designs deviated 1.58-fold, with wide confidence intervals.

Conclusions:

• MSM-studies may produce different conclusions than RCTs.
• Real-world evidence from MSM-studies requires careful interpretation for healthcare decisions.
• Further research is needed to refine causal inference methods in nonrandomized studies.