Improving Inverse Probability Weighting by Post-calibrating Its Propensity Scores.

Area of Science:

• Causal Inference
• Statistical Modeling
• Machine Learning

Background:

• Propensity scores are crucial for causal inference but may not behave as true probabilities.
• Flexible estimators can produce scores with poor calibration, impacting causal effect estimation.
• Existing theoretical guarantees rely on propensity scores acting as conditional probabilities.

Purpose of the Study:

• To evaluate the impact of propensity score calibration on causal inference accuracy.
• To assess the effectiveness of a postprocessing calibration method.
• To compare calibration improvements across different propensity score estimators.

Main Methods:

• A simulation study was conducted to estimate the average treatment effect.
• Propensity scores were generated using various statistical estimators.
• A postprocessing calibration method was applied to the propensity scores.
• Estimation error was measured before and after calibration.

Main Results:

• Propensity score calibration significantly reduced the error in average treatment effect estimation.
• Greater improvements in score calibration led to greater reductions in estimation error.
• Expressive tree-based estimators showed larger relative improvements after calibration compared to logistic regression models.

Conclusions:

• Postprocessing calibration is a computationally inexpensive and effective method to improve causal inference.
• Adopting propensity score calibration is recommended when using expressive models for estimation.
• Calibration enhances the reliability of propensity scores as conditional probabilities for causal inference.