Related Experiment Video
Updated: Jul 2, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Large Language Models for Ophthalmology Training in China: A Prospective Evaluation
Zuhui Zhang1, Changke Huang1, Xinxin Yu1
1National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China.
Ophthalmology Science
|July 1, 2026
Summary
Large language models (LLMs) show promise in ophthalmology training, significantly improving resident performance on text-based exams. However, LLMs struggle with image diagnostics, highlighting potential risks in AI-assisted medical education.
Area of Science:
- Ophthalmology
- Artificial Intelligence
- Medical Education
Background:
- Global shortage and uneven distribution of ophthalmologists necessitate innovative training solutions.
- Large Language Models (LLMs) offer potential for scalable medical education.
- Evaluating LLM effectiveness and risks in ophthalmic training is crucial.
Purpose of the Study:
- To assess the effectiveness of LLMs in ophthalmic training.
- To explore the potential risks associated with LLM use in ophthalmology.
- To investigate LLMs as a solution for ophthalmologist shortages.
Main Methods:
- Prospective study involving 11 LLMs and 10 resident physicians (RPs).
- LLMs tested on Chinese and English ophthalmology qualification exams (CNHPTQE-O).
- Best-performing LLM assisted RPs in text-based exams and keratitis image classification.
Main Results:
- Chinese LLMs, particularly ERNIE Bot 4.5 Turbo, excelled in CNHPTQE-O exams (98.00% Chinese, 86.50% English).
- LLM assistance improved RP accuracy on text exams from 60.75% to 79.00% (P = 0.001).
- LLM assistance did not improve RP accuracy in keratitis image classification (41.25% vs. 40.56%, P = 0.662).
Conclusions:
- LLMs demonstrate strong ophthalmic knowledge and potential as text-based training aids.
- LLM limitations exist in image-assisted diagnostics, posing risks of "artificial ignorance".
- Careful integration of LLMs is needed to maximize benefits and mitigate risks in ophthalmic training.
