Related Experiment Videos
Edoardo Leo1, Francesco Baglivo1, Federico Starace1
1Dipartimento di Ricerca Traslazionale e delle Nuove Tecnologie in Medicina e Chirurgia, Università di Pisa.
Recenti Progressi in Medicina
|October 2, 2025
Abstract
None:
Using Sleep Medicine guidelines and textbook, we evaluated four large language models (LLMs) (Llama 3.2 3B, Llama 3.3 70B, GPT 4o mini, Gemini 2.0 Flash) on AIMS certification questions, comparing baseline and Retrieval Augmented Generation (RAG) performance. RAG improved accuracy in all models (e.g., Llama 3.2 +9.6 points, Gemini 2.0 +4.0 points), highlighting RAG's role in enhancing LLM reliability in specialized medical domain.