Estimating the Distribution of Treatment Effects From Random Design Experiments

Area of Science:

• Statistics
• Econometrics
• Causal Inference

Background:

• Randomized experiments effectively estimate average treatment effects.
• Evaluators often require understanding the full distribution of treatment effects, including benefits, neutral effects, and harms.

Purpose of the Study:

• To explain and illustrate a Bayesian approach for estimating the distribution of treatment effects, as recommended by Imbens and Rubin (I&R).
• To provide computing algorithms for various outcome types (continuous, binary, ordered, countable).

Main Methods:

• Utilizing a Bayesian framework based on Imbens and Rubin's recommendations.
• Developing and applying algorithms for diverse outcome data.
• Illustrating the approach with simulated and real-world data.

Main Results:

• The Bayesian approach yields bounded uncertainty intervals for summary measures of treatment effect distributions.
• These bounds are generally informative, providing useful insights despite potential identification challenges.

Conclusions:

• Bounded solutions offer valuable insights into treatment effect distributions, even with identification issues.
• Evaluators are encouraged to incorporate distribution of treatment effects analyses into both new and completed studies.