Utilizing ChatGPT-3.5 to Assist Ophthalmologists in Clinical Decision-making.

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Diagnostics

Background:

• Artificial intelligence, specifically large language models like ChatGPT-3.5, shows promise in healthcare applications.
• The utility of AI in generating differential diagnoses for ocular pathologies requires investigation.

Purpose of the Study:

• To evaluate the potential of ChatGPT-3.5 in assisting ophthalmologists with differential diagnoses.
• To assess the impact of varying levels of clinical information on the accuracy of AI-generated diagnoses.

Main Methods:

• One hundred ocular pathologies were input into ChatGPT-3.5 using a clinical vignette template with two signs and two symptoms (Method A).
• A subset of 30 pathologies was re-evaluated with additional risk factors (age, sex, past medical history) (Method B).
• Statistical analysis using a one-tailed Wilcoxon signed-rank test compared accuracy between Method A and Method B.

Main Results:

• ChatGPT-3.5 achieved a 51% accuracy for the first differential diagnosis and an 18% accuracy for other differential diagnoses.
• 31% of cases were not included in the AI-generated differential diagnosis list.
• Only neuro-ophthalmology cases showed a statistically significant improvement in diagnostic accuracy with the addition of risk factors (P = 0.01).

Conclusions:

• ChatGPT-3.5 can serve as a supplementary tool for clinicians by suggesting potential diagnoses.
• The current accuracy of ChatGPT-3.5 is insufficient for replacing clinical decision-making in ophthalmology.
• Further research is needed to enhance AI diagnostic capabilities in ophthalmology.