An AI-Driven System for Travel Vaccination Guidance using Agentic Retrieval Augmented Generation (RAG)
Related Experiment Video
Updated: Nov 8, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Luca Di Palma1, Claudio Droghetti1, Katia Iaccarino1
1Department of Diagnostic Imaging and Stereotactic Radiosurgery, Centro Diagnostico Italiano S.p.A., Via S. Saint Bon 20, Milan, 20147, Italy.
Background:
Travel vaccination information is distributed across national health authorities. We developed a system combining conversational and structured access to destination-specific information.
Methods:
A multi-agent chatbot and dashboard integrated information from Viaggiare Sicuri, TravelHealthPro, and CDC Travelers' Health. One physician formulated and scored ten Italian-language queries, evaluating the complete user-visible interaction. Dashboard extraction was assessed across twenty countries, ten purposively selected and ten randomly selected, against a reference combining entries from the same sources. Information-retrieval times were compared within countries for one clinician using paired analyses.
Results:
Mean chatbot scores were 4.9/5 for accuracy and completeness and 4.8/5 for clarity. The dashboard identified 286 of 295 harmonized reference entries, with 97.40% mean country-level recall, 96.95% pooled recall, 100% precision, and 98.56% mean country-level F1-score. Mean retrieval times were 80.9 seconds for manual consultation and 19.7 seconds for the dashboard. The mean paired saving was 61.3 seconds (95% CI: 54.1-68.4; p < 0.001), corresponding to a relative reduction of 75.7% (95% CI: 73.0-78.3%).
Conclusions:
The findings support technical feasibility and preliminary extraction performance. Independent assessment of delivered answers and validation in clinical workflows are needed to establish clinical utility.
