Combining machine learning and matching techniques to improve causal inference in program evaluation.

Area of Science:

• Biostatistics
• Machine Learning in Health Research
• Program Evaluation

Background:

• Program evaluations commonly use matching to mimic randomization for group assignment.
• Covariate balance assessment and treatment effect estimation typically follow matching implementation.
• Existing methods may have limitations with data skewness, outliers, or variable types.

Purpose of the Study:

• Introduce a novel analytic framework using Optimal Discriminant Analysis (ODA) for covariate balance assessment and treatment effect estimation post-matching.
• Highlight ODA's advantages: applicability to diverse metrics and groups, insensitivity to data issues, and use of universal accuracy measures.
• Extend ODA's utility to studies employing analytic weights for adjustment or precise outcome measurement.

Main Methods:

• Employed one-to-one matching on the propensity score as the primary matching strategy.
• Assessed covariate balance using both standardized difference in means (conventional) and ODA's classification accuracy measures.
• Estimated treatment effects via ordinary least squares regression and ODA.

Main Results:

• Empirical data analysis demonstrated that ODA yielded results highly consistent with conventional methods for covariate balance assessment.
• Treatment effect estimations using ODA were also consistent with traditional ordinary least squares regression.
• ODA's accuracy measures proved effective across all prognostic analyses.

Conclusions:

• Combining ODA with matching techniques within a treatment effects framework produces results comparable to conventional approaches.
• ODA presents a compelling alternative due to its added analytical dimensions and robustness.
• The framework's ability to accept analytic weights broadens its applicability in complex study designs.