A Virtual Assistant in Vaccine Pharmacovigilance: Content and Usability Validation

Area of Science:

• Pharmacovigilance
• Health Informatics
• Artificial Intelligence in Healthcare

Background:

• Vaccine pharmacovigilance is crucial for monitoring vaccine safety.
• Traditional methods can be resource-intensive and may have limitations in real-time data collection.
• Developing innovative tools is essential to improve vaccine safety surveillance.

Purpose of the Study:

• To describe the development and validation of a virtual assistant for vaccine pharmacovigilance.
• To assess the content validity and usability of the virtual assistant prototype.

Main Methods:

• Content validation by 22 healthcare professionals.
• Usability testing with 126 users using the System Usability Scale.
• Statistical analysis included agreement rate, content validity index, and kappa (κ) test for interrater reliability.

Main Results:

• Content domains achieved excellent suitability, relevance, and representativeness (>86%).
• Content validity index averaged 0.90 (range 0.81-0.98); interrater reliability was high (κ=0.76).
• Overall usability score was 80.1, with higher scores in users who experienced vaccine reactions (P=.002).

Conclusions:

• The virtual assistant for vaccine pharmacovigilance achieved satisfactory content validity and usability.
• This tool shows promise for enhancing vaccine safety surveillance and patient monitoring.
• The virtual assistant can increase credibility in reporting and managing adverse events following immunization.