Measuring the effect of policy interventions at the population level: some methodological concerns

Health policy evaluations estimate the response of population aggregate outcomes to interventions. However, clarity on the form of the expected causal relationship, the parameter identification strategy, and the mode of hypothesis testing is required to overcome a number of conceptual and methodological problems. We use the New Jersey statewide smoking ban as an example. We examine statewide admission rates for acute myocardial infarctions, strokes and lower limb fractures, and emergency room encounter rates for asthma exacerbations before and after the smoking ban. We discuss the identification options and show the sensitivity of estimates of the response function to different specifications of the stochastic and intervention components and to different modes of inference. Model misspecification is demonstrated by rolling Chow tests for structural breaks in repeated observations.