Cox regression using a calendar time scale was unbiased in simulations of COVID-19 vaccine effectiveness & safety

Observational studies on corona virus disease 2019 (COVID-19) vaccines compare event rates in vaccinated and unvaccinated person time using Poisson or Cox regression. In Cox regression, the chosen time scale needs to account for the time-varying incidence of severe acute respiratory syndrome corona virus 2 (SARS-CoV-2) infection and COVID-19 vaccination.We aimed to quantify bias in person-time based methods, with and without adjustment for calendar time, using simulations and empirical data analysis.

We simulated 500,000 individuals who were followed for 365 days, and a point exposure resembling COVID-19 vaccination (cumulative incidence 80%). We generated an effectiveness outcome, emulating the incidence of severe acute respiratory syndrome corona virus 2 infection in Denmark during 2021 (risk 10%), and a safety outcome with seasonal variation (myocarditis, risk 1/5,000). Incidence rate ratios (IRRs) were set to 0.1 for effectiveness and 5.0 for safety outcomes. IRRs and hazard ratios (HRs) were estimated using Poisson and Cox regression with a time under observation scale, and a calendar time scale. Bias was defined as estimated IRR or HR-true IRR. Further, we obtained estimates for both outcomes using data from the Danish health registries.

Unadjusted IRRs (bias_effectivenes +0.16; bias_safety -2.09) and HRs estimated using a time-under-observation scale (+0.28;-2.15) were biased. Adjustment for calendar time reduced bias in Cox (+0.03; +0.33) and Poisson regression (0.00; -0.28). Cox regression using a calendar time scale was least biased (0.00, +0.12). When analyzing empirical data, adjusted Poisson and Cox regression using a calendar time scale yielded estimates in accordance with existing evidence.

Lack of adjustment for the time-varying incidence of COVID-19 related outcomes may severely bias estimates.