The impact of selection bias on vaccine effectiveness estimates from test-negative studies

Estimates of vaccine effectiveness (VE) from test-negative studies may be subject to selection bias. In the context of influenza VE, we used simulations to identify situations in which meaningful selection bias can occur. We also analyzed observational study data for evidence of selection bias.

For the simulation study, we defined a hypothetical population whose members are at risk for acute respiratory illness (ARI) due to influenza and other pathogens. An unmeasured "healthcare seeking proclivity" affects both probability of vaccination and probability of seeking care for an ARI. We varied the direction and magnitude of these effects and identified situations where meaningful bias occurred. For the observational study, we reanalyzed data from the United States Influenza VE Network, an ongoing test-negative study. We compared "bias-naïve" VE estimates to bias-adjusted estimates, which used data from the source populations to correct for sampling bias.

In the simulation study, an unmeasured care-seeking proclivity could create selection bias if persons with influenza ARI were more (or less) likely to seek care than persons with non-influenza ARI. However, selection bias was only meaningful when rates of care seeking between influenza ARI and non-influenza ARI were very different. In the observational study, the bias-naïve VE estimate of 55% (95% CI, 47--62%) was trivially different from the bias-adjusted VE estimate of 57% (95% CI, 49--63%).

In combination, these studies suggest that while selection bias is possible in test-negative VE studies, this bias in unlikely to be meaningful under conditions likely to be encountered in practice. Researchers and public health officials can continue to rely on VE estimates from test-negative studies.