Estimating effects of health policy interventions using interrupted time-series analyses: a simulation study

Area of Science:

• Epidemiology
• Biostatistics
• Health Services Research

Background:

• Interrupted time-series (ITS) analysis is a quasi-experimental design commonly used to evaluate health policy interventions.
• This method utilizes pre- and post-intervention data without randomization to assess policy impacts.
• A simulation-based approach was employed to estimate intervention effects under various assumptions.

Purpose of the Study:

• To compare the accuracy of different analytical approaches in interrupted time-series (ITS) analysis for health policy evaluation.
• To investigate the impact of model misspecification on the estimation of intervention effects.
• To assess the influence of intervention timing on the reliability of ITS analyses.

Main Methods:

• Simulated mortality rates incorporated linear trends, seasonality, autoregressive, and moving-average terms.
• Policy effects were simulated under three scenarios: immediate-level change, immediate-level and slope change, and lagged-level and slope change.
• Generalized additive mixed models were used, comparing effects derived from estimated coefficients versus model predictions, under both correct and misspecified model conditions.

Main Results:

• When models were correctly specified, both estimated and predicted approaches yielded similar results.
• Model misspecification led to significant discrepancies between the two approaches, with the predicted approach providing estimates closer to the true intervention effect.
• The difference between approaches was more pronounced when the policy intervention occurred earlier in the time-series.

Conclusions:

• Caution is advised in interrupted time-series (ITS) analyses, as statistical power depends on intervention timing within the series, not just sample size.
• The consideration of lagged intervention effects is crucial during the study design phase, particularly when developing statistical models.
• The choice of analytical approach (estimated coefficients vs. model predictions) significantly impacts the reliability of health policy intervention effect estimation, especially under model misspecification.