Characteristics of Vaccine Safety Observational Studies and Authors' Attitudes: A Systematic Review

Postlicensure observational studies are the mainstay of vaccine safety evaluation. However, these studies have well-known methodological limitations, rendering them particularly vulnerable to unmeasured confounding. We sought to describe high-impact observational studies of vaccine safety, investigate the authors' attitudes toward their study's findings and limitations, and report on spin practices.

We conducted a PubMed systematic review of comparative observational studies of vaccine safety published in the 6 top medical journals from inception to March 2024.

Thirty-seven studies were included, spanning publications from 1995 to 2024. Most studies focused on COVID-19 and influenza vaccines (n = 11, 30%, and n = 10, 27%, respectively). Study designs and methodologies varied. Electronic health records (54%), passive surveillance databases (32%), and national registries (27%) were the most common data sources. Negative control outcomes were used in a single study. Residual confounding was conceded in 54% of studies, and an additional 24% did so implicitly. Spin was noted in 48.6% of the studies. This systematic review found that authors of observational vaccine safety studies in high-impact medical journals often acknowledge residual confounding, but rarely use methods like negative control outcomes to better detect unmeasured confounding. Furthermore, spin is common, occurring in approximately 50% of the studies.

Although our findings are somewhat limited by subjectivity in study assessments, they suggest that editors and reviewers of high-impact journals should ensure that the language used in reporting observational studies accurately reflects the findings and their limitations.