Related Experiment Videos
Guido Marchi1, Giulia Gambini2, Giacomo Guglielmi3
1UO Pneumologia, Dipartimento Cardio-Toraco-Vascolare, Azienda ospedaliero-universitaria Pisana, Pisa.
Recenti Progressi in Medicina
|October 2, 2025
Abstract:
Three LLMs - ChatGPT-4, Claude 3.5 Sonnet and Gemini 1.5 Advanced - were evaluated on COPD questions from the GOLD recommendations. Sixty-one pulmonologists from 6 continents rated 90 AI responses for completeness, accuracy, terminology, accessibility, and safety. Gemini outperformed in completeness, Claude in accuracy and terminology, with no differences in accessibility or safety. While promising, clinical use requires caution and further validation to ensure safe, accurate patient education.