What randomization can and cannot guarantee

Area of Science:

• Statistics
• Causal Inference
• Epidemiology

Background:

• The value of randomized controlled trials (RCTs) in causal inference is debated.
• Robins and Ritov (1997) established key results regarding statistical inference in RCTs versus observational studies.
• Pearl and Mackenzie (2018) controversially claimed RCTs are not the gold standard for causal analysis.

Purpose of the Study:

• To clarify the statistical inference advantages of RCTs over unconfounded observational studies.
• To address and refute the claim that RCTs are not the gold standard in causal analysis.
• To provide a broader review of randomization's guarantees and limitations in causal inference.

Main Methods:

• Review and delineation of key results from Robins and Ritov (1997).
• Comparative analysis of statistical inference in randomized controlled trials versus unconfounded observational studies.
• Discussion of randomization-based inference and its limitations.

Main Results:

• Randomized controlled trials (RCTs) provide significantly stronger statistical inference than unconfounded observational studies.
• Nonparametric identification of causal effects is identical in both RCTs and observational studies, yet inference differs.
• Claims that RCTs are not the gold standard are misleading for finite sample causal effect estimation.

Conclusions:

• Randomization in RCTs is crucial for robust causal effect estimation, especially with finite sample sizes.
• While randomization is valuable, it has limitations for complex causal inference questions.
• Sensitivity analysis remains important for strengthening causal claims from observational studies.