Interrupted Time Series Design and Analyses in Health Policy Assessment

Interrupted time series design is a quasi-experimental study design commonly used to evaluate the impact of a particular intervention (e.g., a health policy implementation) on a specific outcome. Two of the most often recommended analytical approaches to interrupted time series analysis are autoregressive integrated moving average (ARIMA) and Generalized Additive Models (GAM). We conducted simulation tests to determine the performance differences between ARIMA and GAM methodology across different policy effect sizes, with or without seasonality, and with or without misspecification of policy variables. We found that ARIMA exhibited more consistent results under certain conditions, such as with different policy effect sizes, with or without seasonality, while GAM were more robust when the model was misspecified. Given these findings, the variation between the models underscores the need for careful model selection and validation in health policy studies.