Evaluating large language models in specialized myopia knowledge and clinical reasoning

Myopia is a major global public health concern, and the growing use of LLMs for health consultation makes the accuracy of their myopia-related responses increasingly important. Existing ophthalmic LLM studies have largely focused on image-based applications and mixed ocular diseases, with limited evaluation of myopia-specific reasoning. Yet myopia-related inquiries span both breadth and depth, from specialized differential considerations to common issues in prevention, control, and public education. This study aimed to assess mainstream LLMs across multiple levels of myopia-related knowledge and clinical reasoning.

This comparative evaluation study assessed six large language models: GPT-5.3 Instant, DeepSeek-V3.2, Gemini 3.1 Pro Preview, Claude Opus 4.7, Qwen3.6-Plus, and Doubao-Seed-2.0-Pro. The assessment covered four clinically relevant domains of myopia care and included hierarchical case-based scenarios spanning core judgment, preferred management, and follow-up. Objective items were submitted in independent new chat sessions, repeated three times, and tested without additional medically enhancing prompts. For the open-ended case tasks, responses from the six models and eight junior physicians were independently scored by two senior ophthalmology experts.

ChatGPT showed the highest accuracy in standardized judgment tasks, case-based clinical tasks, and external validation multiple-choice questions, achieving 92.33% ± 1.15%, 85.00% ± 1.67%, and 88.00% ± 1.00%, respectively, with DeepSeek ranking second. In case-based scenarios, model performance was strongest for core judgment but declined notably for preferred management decisions. In the open-ended case assessment, ChatGPT, DeepSeek, Claude, and Gemini achieved higher expert-rated scores than junior clinicians, whereas Doubao and Qwen showed lower overall performance. Across the four clinical domains, performance differences widened in tasks requiring applied management and referral reasoning.

Current LLMs demonstrated substantial capability in myopia-related knowledge assessment and text-based clinical reasoning. For patients and the public, these models may provide convenient access to relatively professional responses to common myopia-related questions. Although several models achieved higher expert-rated scores than junior clinicians in standardized written cases, this advantage was limited to text-based assessment and should not be extrapolated to real-world clinical performance. Given the complexity of specialist judgment and individual variation, model-generated responses should serve as informative references for consultation, health education, and preliminary decision support, not substitutes for formal clinical advice or physician-led individualized management.