Challenges in Obtaining Valid Causal Effect Estimates with Machine Learning Algorithms.

Area of Science:

• Causal inference
• Statistical modeling
• Machine learning applications

Background:

• Machine learning (ML) methods are increasingly proposed for causal effect estimation due to their flexibility.
• However, ML algorithms can sometimes underperform compared to traditional parametric regression.
• The performance of ML-based estimators, particularly single- and double-robust types, requires thorough investigation.

Purpose of the Study:

• To evaluate the performance of ML-based single- and double-robust estimators for causal effect estimation.
• To compare these methods against parametric regression under varying confounding scenarios.
• To identify conditions and techniques that improve the reliability of ML-based causal inference.

Main Methods:

• Conducted 100 Monte Carlo simulations with sample sizes of 200, 1200, and 5000.
• Investigated bias and confidence interval coverage in simple and complex confounding scenarios.
• Assessed ML algorithms within single- and double-robust estimation frameworks, including techniques like sample splitting and confounder interactions.

Main Results:

• In simple confounding, double-robust ML estimators outperformed single-robust ones.
• In complex nonlinear confounding, single-robust ML estimators showed significant bias, similar to misspecified parametric models.
• While double-robust estimators were less biased in complex scenarios, coverage remained suboptimal unless sample splitting, interactions, and rich ML models were used.

Conclusions:

• ML-based singly robust methods for causal inference are not recommended due to potential bias.
• A combination of doubly robust estimation, sample splitting, confounder interactions, and richly specified ML algorithms is essential for reliable causal effect estimation.
• Careful implementation is necessary to harness the benefits of ML in causal inference.