What can GPT-4 do for Diagnosing Rare Eye Diseases? A Pilot Study

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Diagnostics

Background:

• Generative Pretrained Transformer-4 (GPT-4) has broad applications, but its utility in ophthalmology remains underexplored.
• Rare ophthalmic diseases present diagnostic challenges, necessitating advanced tools for timely identification.

Purpose of the Study:

• To evaluate GPT-4's capability in identifying rare ophthalmic diseases across simulated patient, family physician, and junior ophthalmologist scenarios.
• To assess the suitability and accuracy of GPT-4's diagnostic suggestions by experienced ophthalmologists.

Main Methods:

• Ten rare ophthalmic disease cases were selected from the EyeRounds service.
• GPT-4 processed increasing amounts of case information to simulate different user scenarios.
• Senior ophthalmologists evaluated GPT-4's responses for appropriateness and accuracy.

Main Results:

• 83.3% of GPT-4's responses were deemed appropriate by senior ophthalmologists.
• Accuracy varied by scenario: 0% for simulated patients, 50% for family physicians, and 90% for junior ophthalmologists.
• GPT-4 provided general disease possibilities for patients but accurate diagnoses for junior ophthalmologists.

Conclusions:

• GPT-4 demonstrates potential as a consultation tool for patients and physicians in identifying rare eye diseases.
• The model can assist junior ophthalmologists in diagnosing complex ophthalmic conditions.
• Clinical implementation requires caution, emphasizing the need for expert verification and judicious referral practices.