Harmonizing Case Validation for Vaccine Safety in Real-World Data: The VAC4EU Experience.

Area of Science:

• Vaccine Safety Research
• Pharmacovigilance
• Real-World Data Analysis

Background:

• Standardized Brighton Collaboration (BC) case definitions are challenging to apply retrospectively to heterogeneous real-world data (RWD) due to inconsistent clinical details and data structures.
• This inconsistency hinders reliable vaccine safety outcome validation in post-authorization studies.

Purpose of the Study:

• To develop and validate a structured pipeline by the Vaccine Monitoring Collaboration for Europe (VAC4EU) to operationalize BC case definitions for vaccine safety outcomes in RWD.
• To harmonize, scale, and ensure reusability of BC case definitions for vaccine safety surveillance in diverse RWD settings.

Main Methods:

• A systematic, stepwise approach was developed, adapting existing BC case definitions or creating new ones for RWD.
• Key steps included expert review, creation of dummy cases, development of REDCap electronic data collection forms (eDCFs) with decision logic for levels of certainty (LOC), iterative testing, and comprehensive abstractor training.
• Inter-rater reliability was assessed using Fleiss' kappa (κ) by comparing abstractor LOCs to a reference standard.

Main Results:

• The pipeline was applied to 16 COVID-19 vaccine safety outcomes, with 13 having existing BC definitions.
• 78 dummy case descriptions and 15 REDCap eDCFs were created; 747 dummy case abstractions were completed by 33 abstractors.
• Moderate overall concordance (κ = 0.55) was observed, though with variability across outcomes, including low agreement for thrombosis with thrombocytopenia syndrome (κ = -0.05).

Conclusions:

• The VAC4EU validation pipeline offers a standardized framework for training and validating vaccine safety outcomes in RWD.
• Adapting BC case definitions and providing dedicated abstractor training are crucial for reducing variability in outcome validation.
• This standardized approach is essential for enhancing the reliability of post-authorization safety studies.