An Exploratory Study of the Impact of COVID-19 Vaccine Spontaneous Reporting on Masking Signal Detection in EudraVigilance

During the signal detection process, statistical methods are used to identify drug-event combinations (DECs) which are disproportionately reported when compared with other drugs and events in the entire database. We hypothesise that the high volume of COVID-19 vaccine adverse drug reaction (ADR) reports transmitted to EudraVigilance may have affected the performance of disproportionality statistics used in routine signal detection, potentially resulting in signals either being masked, or false associations being flagged as potential signals.

Our aim was to study the impact of COVID-19 vaccine spontaneous reporting on statistical signal detection in EudraVigilance.

We recalculated the reporting odds ratio (ROR) for signals that were previously discussed at the level of the Pharmacovigilance Risk Assessment Committee, or signals that were retrieved from EudraVigilance, by omitting COVID-19 vaccine reports from the standard ROR calculation and then comparing the lower confidence interval (LCI) of the recalculated ROR to the LCI of the actual ROR in EudraVigilance.

In total, 52 signals for 38 active substances were reviewed. For 35 signals, the LCI of the recalculated ROR value was lower than the LCI of the actual ROR (suggesting that COVID-19 vaccine ADR reporting had a positive effect on the strength of the signal) while for 15 signals the LCI of the recalculated ROR value was higher than the LCI of the actual ROR (suggesting that COVID-19 vaccine ADR reporting had an attenuating effect on the strength of the signal). For two signals, no change in the ROR was observed. In our analysis, six significant results were found. Five DECs were found to be masked: bleomycin and immune thrombocytopenia (actual ROR LCI = 0.94, recalculated ROR LCI = 1.02), vortioxetine and heavy menstrual bleeding (actual ROR LCI = 0.3, recalculated ROR LCI = 1.06), caplacizumab and heavy menstrual bleeding (actual ROR LCI = 0.98, recalculated ROR LCI = 3.47), ziprasidone and amenorrhoea (actual ROR LCI = 0.84, recalculated ROR LCI = 1.67), and azacitidine and pericarditis (actual ROR LCI = 0.81, recalculated ROR LCI = 2.01). For the DEC of adalimumab and immune reconstitution inflammatory syndrome, the LCI of the actual ROR value was 1.14 and removing COVID-19 vaccine reporting resulted in an LCI of the recalculated ROR value of 0.94 (below threshold).

We demonstrated five cases of masking and one case of false-positive association due to the influence of COVID-19 vaccine spontaneous reporting on the ROR. This suggests that the high number of adverse drug reaction reports for COVID-19 vaccines in EudraVigilance has the potential to affect routine statistical signal detection activities. The impact of COVID-19 vaccine ADR reports on current signal detection practices requires further evaluation and solutions to tackle masking issues in EudraVigilance may need to be developed.