Related Experiment Video
Updated: Jun 1, 2025
A High-throughput Platform for the Screening of Salmonella spp./Shigella spp.
Published on: November 7, 2018
Performance of ChatGPT-4o in the diagnostic workup of fever among returning travellers requiring hospitalization: a
Dana Yelin1,2, Neta Shirin1, Itai Harris3
1Infectious Diseases Unit, Sheba Medical Center, Tel HaShomer, Ramat Gan, Israel.
Background:
Febrile illness in returned travellers presents a diagnostic challenge in non-endemic settings. Chat generative pretrained transformer (ChatGPT) has the potential to assist in medical tasks, yet its diagnostic performance in clinical settings has rarely been evaluated. We conducted a validation assessment of ChatGPT-4o's performance in the workup of fever in returning travellers.
Methods:
We retrieved the medical records of returning travellers hospitalized with fever during 2009-2024. Their clinical scenarios at time of presentation to the emergency department were prompted to ChatGPT-4o, using a detailed uniform format. The model was further prompted with four consistent questions concerning the differential diagnosis and recommended workup. To avoid training, we kept the model blinded to the final diagnosis. Our primary outcome was ChatGPT-4o's success rates in predicting the final diagnosis when requested to specify the top three differential diagnoses. Secondary outcomes were success rates when prompted to specify the single most likely diagnosis, and all necessary diagnostics. We also assessed ChatGPT-4o as a predicting tool for malaria and qualitatively evaluated its failures.
Results:
ChatGPT-4o predicted the final diagnosis in 68% [95% confidence interval (CI) 59-77%], 78% (95% CI 69-85%) and 83% (95% CI 74-89%) of the 114 cases, when prompted to specify the most likely diagnosis, top three diagnoses and all possible diagnoses, respectively. ChatGPT-4o showed a sensitivity of 100% (95% CI 93-100%) and a specificity of 94% (95% CI 85-98%) for predicting malaria. The model failed to provide the final diagnosis in 18% (20/114) of cases, primarily by failing to predict globally endemic infections (16/21, 76%).
Conclusions:
ChatGPT-4o demonstrated high diagnostic accuracy when prompted with real-life scenarios of febrile returning travellers presenting to the emergency department, especially for malaria. Model training is expected to yield an improved performance and facilitate diagnostic decision-making in the field.
Related Concept Videos
01:30Pneumonia III: Complications and Assessment
01:26Pulmonary Tuberculosis IV
Several diagnostic approaches are used to detect TB. The conventional method is the Tuberculin Skin Test (TST), also known as the Mantoux test. However, this method has...
01:28Pneumonia IV: Management
Bacterial Pneumonia Treatment
For bacterial pneumonia, antibiotics serve as the cornerstone of therapy. Initial treatment often begins with empirical antibiotics, tailored to the anticipated causative organism and adjusted based on culture results. Key antibiotic choices include:
01:25Increased Body Temperature
01:17Transmission-based Precautions I: Contact, Enteric, and Droplets
Contact Precautions:
Contact precautions are the measures taken to prevent the transmission of infectious agents, especially epidemiologically important microorganisms such as MRSA or influenza, primarily transmitted through direct or indirect contact with an...
01:26Patterns of Fever