Using forecast modelling to evaluate treatment effects in single-group interrupted time series analysis

Area of Science:

• Epidemiology
• Biostatistics
• Public Health Policy Evaluation

Background:

• Interrupted time series analysis (ITSA) is a common method for evaluating interventions.
• Traditional ITSA methods can produce biased results if model assumptions are violated.
• Accurate causal inference in single-group ITSA is challenging.

Purpose of the Study:

• To assess treatment effects in single-group ITSA using forecasting methods.
• To compare forecasting approaches with traditional ITSA for evaluating interventions.
• To improve causal inference in ITSA studies.

Main Methods:

• Employed forecasting models (linear regression, Holt-Winters, ARIMA) to fit pre-intervention data.
• Generated post-intervention forecasts to establish a counterfactual trend.
• Compared actual post-intervention observations with forecasted trends to infer treatment effects.
• Demonstrated the approach using California's Proposition 99 on cigarette sales.

Main Results:

• Holt-Winters and ARIMA models provided the best fit for pre-intervention data.
• Linear regression poorly fit the pre-intervention data.
• Post-intervention observations were above forecasts for Holt-Winters and ARIMA, suggesting no effect.
• Post-intervention observations were below forecasts for linear regression, suggesting a potential effect, highlighting model sensitivity.

Conclusions:

• Forecasting models offer a robust alternative for fitting pre-intervention data in ITSA.
• Accurate counterfactual forecasts are crucial for improving causal inference in single-group ITSA.
• Model misspecification can lead to biased treatment effect estimations in ITSA.