Context-Aware Vision Language Foundation Models for Ocular Disease Screening in Retinal Images

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Ophthalmology

Background:

• Foundation models offer adaptable representations for medical imaging challenges.
• Vision-language foundation (VLF) models are emerging for image analysis.
• Ocular disease screening faces data variability and domain shift issues.

Purpose of the Study:

• To advance VLF models for ocular disease screening using the comprehensive OPHDIAT dataset.
• To develop context-aware VLF models leveraging multi-image and clinical data.
• To enhance model robustness against domain shifts in retinal imaging.

Main Methods:

• Utilized the OPHDIAT dataset with nearly 700,000 fundus photographs and extensive clinical data.
• Developed novel context-aware VLF models building upon the FLAIR architecture.
• Evaluated models on in-domain (OPHDIAT) and out-of-domain public datasets.

Main Results:

• Achieved improved in-domain diabetic retinopathy (DR) grading with AUCs from 0.851 to 0.9999.
• Demonstrated strong generalization for ocular disease detection on out-of-domain data (AUC: 0.631-0.913).
• Context-aware VLF models showed enhanced performance by analyzing multiple images and clinical context.

Conclusions:

• Context-aware VLF models significantly enhance DR grading accuracy and robustness.
• These models show potential for effective, generalized ocular disease screening.
• Leveraging rich datasets and contextual information is key for advancing medical AI.