Specialized curricula for training vision language models in retinal image analysis.

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Imaging Analysis

Background:

• Clinicians face significant time burdens reviewing medical images and transcribing findings.
• Foundation models integrating visual and textual data show potential for reducing workloads but clinical value is uncertain.
• Current general and medical vision-language models (VLMs) underperform ophthalmologists in age-related macular degeneration (AMD) tasks.

Purpose of the Study:

• To develop and evaluate a specialized training curriculum to optimize VLMs for clinical decision-making in ophthalmology.
• To assess the performance of a curriculum-enhanced VLM against general foundation models and human experts in AMD-related tasks.

Main Methods:

• A dedicated training curriculum was designed by domain specialists to optimize VLMs for clinical decision-making.
• The performance of the specialized model (RetinaVLM-Specialist) was compared against foundation medical VLMs and ChatGPT-4o in AMD disease staging and referral tasks.
• A reader study involving senior ophthalmologists evaluated the accuracy of reports generated by RetinaVLM-Specialist versus ChatGPT-4o.

Main Results:

• RetinaVLM-Specialist significantly outperformed foundation medical VLMs and ChatGPT-4o in AMD disease staging (F1: 0.63 vs. 0.33) and referral (0.67 vs. 0.50).
• The specialized VLM achieved performance comparable to junior ophthalmologists.
• Senior ophthalmologists found RetinaVLM-Specialist reports substantially more accurate than ChatGPT-4o reports (64.3% vs. 14.3%).

Conclusions:

• A curriculum-based training approach can effectively adapt foundation models for specialized medical applications.
• Optimized VLMs demonstrate potential to assist clinicians in real-world medical tasks, improving accuracy and efficiency.
• This study provides a blueprint for enhancing AI models in ophthalmology and other medical fields.