A multimodal retinal image dataset for diabetic retinopathy detection using foundation models.

Area of Science:

• Ophthalmology and Artificial Intelligence
• Medical Imaging and Diagnostics

Background:

• Diabetic Retinopathy (DR) is a major cause of preventable blindness globally.
• Existing deep learning models for DR diagnosis are hindered by limited, small-scale public datasets lacking fine-grained annotations.
• This dataset limitation compromises the reliable assessment of AI model generalizability and real-world clinical utility.

Purpose of the Study:

• To introduce a comprehensive, large-scale multimodal dataset for evaluating AI diagnostic tools for DR and Diabetic Macular Edema (DME).
• To establish a foundational benchmark for assessing AI model performance, generalizability, and domain-specific challenges in retinal imaging.

Main Methods:

• Curated a multimodal dataset encompassing Color Fundus Photography (CFP), Optical Coherence Tomography (OCT), and Ultrawide-field Fundus Imaging (UWF).
• Provided detailed, lesion-level annotations and severity grades for DR and DME across all imaging modalities.
• Benchmarked various fundus foundation models and large vision-language models on the newly created dataset.

Main Results:

• The dataset is unprecedented in scale and modality diversity for DR and DME research.
• Benchmarking revealed critical performance gaps and domain-specific challenges for current AI models.
• The unified, large-scale multimodal data with precise clinical labels serves as a robust evaluation benchmark.

Conclusions:

• The developed multimodal dataset significantly advances the evaluation of AI diagnostic tools for DR and DME.
• This resource is crucial for improving AI reliability and facilitating its translation into clinical practice.
• Future AI development and validation for retinal diseases can leverage this foundational benchmark.