How Three-Arm Random Assignment Within Sites can Improve Non-Experimental Cross-Site Estimates of the Relationship...

Area of Science:

• Program evaluation
• Econometrics
• Causal inference

Background:

• Program evaluations often face bias from unobserved factors.
• Attributing impacts to specific program components is challenging.
• Existing methods may not adequately address cross-site variations.

Purpose of the Study:

• Introduce and evaluate the Cross-Site Attributional Model Improved by Calibration to Within-Site Individual Randomization Findings (CAMIC).
• Reduce bias in estimating program impacts attributed to design, implementation, and context.
• Provide a methodological framework for multi-site experiments.

Main Methods:

• CAMIC utilizes multi-site experiments with three arms: standard treatment, enhanced treatment, and control.
• Requires randomization of individuals within participating sites.
• Employs calibration to within-site individual randomization findings.

Main Results:

• CAMIC effectively reduces bias in attributing program impacts.
• The method is particularly promising when program enhancements correlate highly with other characteristics.
• Experimental estimates of enhancements improve bias reduction for non-randomized characteristics.

Conclusions:

• CAMIC offers a robust approach for disentangling program component effects in multi-site studies.
• The Health Profession Opportunity Grants (HPOG) program evaluation serves as a relevant example.
• CAMIC has broad applicability for future program evaluations seeking precise impact attribution.