Compared with what? Estimating the effects of injury prevention policies using the synthetic control method

This paper discusses the application of the synthetic control method to injury-related interventions using aggregate data from public information systems. The method selects and determines the optimal control unit in the data by minimising the difference between the pre-intervention outcomes in one treated unit (eg, a state) and a weighted combination of potential control units.

I demonstrate the synthetic control method by an application to Florida's post-2010 policy and law enforcement initiatives aimed at bringing down opioid overdose deaths. Using opioid-related mortality data for a panel of 46 states observed from 1999 to 2015, the analysis suggests that a weighted combination of Maine (46.1%), Pennsylvania (34.4%), Nevada (5.4%), Washington (5.3%), West Virginia (4.3%) and Oklahoma (3.4%) best predicts the preintervention trajectory of opioid-related deaths in Florida between 1999 and 2009. Model specification and placebo tests, as well as an iterative leave-k-out sensitivity analysis are used as falsification tests.

The results indicate that the policies have decreased the incidence of opioid-related deaths in Florida by roughly 40% (or -6.19 deaths per 100.000 person-years) by 2015 compared with the evolution projected by the synthetic control unit. Sensitivity analyses yield an average estimate of -4.55 deaths per 100.000 person-years (2.5th percentile: -1.24, 97.5th percentile: -7.92). The estimated cumulative effect in terms of deaths prevented in the postperiod is 3705 (2.5th percentile: 1302, 97.5th percentile: 6412).

Recommendations for practice, future research and potential pitfalls, especially concerning low-count data, are discussed. Replication codes for Stata are provided.