Interpretation of Clinical Retinal Images Using an Artificial Intelligence Chatbot

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Diagnostics

Background:

• Ophthalmic diagnostics increasingly utilize artificial intelligence (AI) tools.
• Evaluating the diagnostic accuracy of advanced AI models like Chat Generative Pre-Trained Transformer-4 (GPT-4) in ophthalmology is crucial.
• OCTCases provides a valuable dataset of multimodal retina teaching cases for performance assessment.

Purpose of the Study:

• To assess the diagnostic performance of Chat GPT-4 in analyzing retina teaching cases from OCTCases.
• To determine the impact of varying amounts of clinical information on AI diagnostic accuracy.
• To investigate the relationship between text-based information and correct diagnosis in multimodal ophthalmic datasets.

Main Methods:

• A cross-sectional study utilizing 69 retina teaching cases from OCTCases.
• Chat GPT-4 was prompted with multimodal ophthalmic images and varying levels of clinical text information.
• Multivariable logistic regression was used to analyze the association between text input and diagnostic accuracy.

Main Results:

• Chat GPT-4 achieved a correct diagnosis in 50.7% of the 69 retina cases.
• A correct diagnosis was achieved in 65.7% of cases when the complete patient description was provided.
• Providing the entire patient description significantly increased the odds of a correct diagnosis (OR=10.1, P < 0.01) compared to minimal information.

Conclusions:

• Chat GPT-4 demonstrates moderate diagnostic accuracy for retina cases with multimodal input, heavily reliant on accompanying text.
• The AI's ability to interpret multimodal imaging without text-based context is currently limited.
• Careful consideration of AI tool application and bioethical implications is essential in clinical practice.