Related Experiment Video
Updated: Jun 12, 2025
05:40
Using an Automated Hirschberg Test App to Evaluate Ocular Alignment
Published on: March 24, 2020
12.2K
Large language model-based multimodal system for detecting and grading ocular surface diseases from smartphone images
Zhongwen Li1,2, Zhouqian Wang1, Liheng Xiu3
1Ningbo Key Laboratory of Medical Research on Blinding Eye Diseases, Ningbo Eye Institute, Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, China.
Frontiers in Cell and Developmental Biology
|June 9, 2025
Summary
A new AI tool, MOSAIC, accurately detects ocular surface diseases using smartphone images. This system shows promise for improving eye care accessibility in underserved regions with minimal training.
Area of Science:
- Ophthalmology
- Artificial Intelligence
- Medical Imaging
Background:
- Medical artificial intelligence (AI) is crucial for addressing healthcare disparities, especially in underserved areas.
- Accessible, interpretable, and automated AI systems are needed to expand quality healthcare.
- Ocular surface diseases (OSDs) require accessible diagnostic tools.
Purpose of the Study:
- To develop and validate the Multimodal Ocular Surface Assessment and Interpretation Copilot (MOSAIC).
- To assess MOSAIC's performance in detecting and grading OSDs using smartphone images.
- To evaluate the interpretability and few-shot learning capabilities of MOSAIC.
Main Methods:
- Developed MOSAIC using gpt-4-turbo, claude-3-opus, and gemini-1.5-pro-latest large language models.
- Utilized 375 smartphone ocular surface images from 290 eyes for validation.
- Evaluated MOSAIC in zero-shot and few-shot settings for image quality control, OSD detection, keratitis grading, and pterygium grading.
Main Results:
- MOSAIC achieved 95.00% accuracy in image quality control and 86.96% in OSD detection.
- Distinguished mild from severe keratitis with 88.33% accuracy and graded pterygium with 66.67% accuracy (five-shot).
- Demonstrated significant performance improvement with increased learning shots (p < 0.01) and high ROUGE-L F1 scores (0.70-0.78).
Conclusions:
- MOSAIC exhibits strong few-shot learning capabilities for OSD management with minimal data.
- The system has high potential for smartphone integration to improve OSD detection and grading in resource-limited settings.
- MOSAIC enhances healthcare accessibility and effectiveness for ocular surface diseases.
