Related Experiment Video
Updated: Apr 25, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
1.3K
Comparative Evaluation of Deep-Reasoning Large Language Models for Ophthalmic Emergencies
Xuanqiao Lin1, Yizhou Yang1, Yuecheng Ren2
1Department of Ophthalmology, Eye, Ear, Nose, and Throat Hospital of Fudan University, Shanghai, People's Republic of China.
Ophthalmology Science
|April 24, 2026
Summary
Large language models (LLMs) show promise in assessing ophthalmic emergencies, with diagnostic accuracy improving with more information. However, their use requires caution due to varied testing recommendations and uncommon ideal safety profiles.
Area of Science:
- Ophthalmology
- Artificial Intelligence
- Medical Diagnostics
Background:
- Ophthalmic emergencies require rapid and accurate assessment.
- Large language models (LLMs) are increasingly capable of processing complex medical information.
- Evaluating LLMs in clinical workflows is crucial for safe integration.
Purpose of the Study:
- To assess the performance of deep-reasoning LLMs in diagnosing ophthalmic emergencies.
- To evaluate LLM accuracy across sequential levels of clinical information.
- To analyze LLM-generated ancillary testing recommendations.
Main Methods:
- A cross-sectional, vignette-based study using 34 ophthalmic emergency cases.
- Cases were presented to six LLMs at three information levels (history, basic exam, specialist exam).
- LLM outputs (diagnosis, triage, testing) were scored by two ophthalmologists.
Main Results:
- Overall triage accuracy was high (85.3%), but diagnostic correctness was moderate (46.9% fully correct).
- Diagnostic accuracy improved with increased information levels (43.1% to 53.9%).
- Ancillary testing recommendations frequently involved undertesting (51.0%) or overtesting (27.5%).
Conclusions:
- Deep-reasoning LLMs show potential for ophthalmic emergency assessment, particularly in triage.
- Diagnostic performance and testing recommendations necessitate careful oversight and guardrails.
- Supervised deployment of LLMs in ophthalmology requires explicit guidelines for workup suggestions.
