Assessing GPT-4o in cataract surgery decision-making: appropriateness, consistency, and clinical implications

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Informatics

Background:

• Cataract surgery involves complex clinical decision-making for intraocular lens (IOL) selection.
• Large language models (LLMs) offer potential for augmenting clinical decision support systems.

Purpose of the Study:

• To evaluate the clinical decision-making capabilities of GPT-4o, a large language model, in selecting intraocular lenses (IOLs) for cataract surgery.
• To compare the appropriateness of LLM-generated IOL recommendations against those made by human surgeons using real-world patient data.

Main Methods:

• A retrospective observational study analyzed data from 74 cataract patients (146 eyes).
• Preoperative data, including imaging, biometrics, and patient history, were input into GPT-4o, prompted to act as an experienced surgeon.
• GPT-4o's IOL recommendations were assessed for appropriateness against an expert consensus and compared with human surgeon choices, analyzing agreement and consistency metrics.

Main Results:

• GPT-4o achieved a significantly lower overall appropriateness rate (37.84%) compared to human surgeons (72.14%) for IOL selection (P=0.001).
• Appropriateness varied by IOL type, with high accuracy for monofocal IOLs (90.0%) but low accuracy for trifocal toric IOLs (9.09%).
• The model exhibited instability in responses and poor inter-rater agreement, though it showed slightly higher consistency (ICC) than human surgeons.

Conclusions:

• GPT-4o demonstrated moderate appropriateness for routine monofocal IOL selection but limited utility in complex, customized cases.
• The findings suggest LLMs could aid in simpler cataract surgery decisions, but require significant domain-specific fine-tuning for advanced applications.
• Further research into multimodal optimization is necessary to enhance LLM performance in complex ophthalmic decision-making.