Related Experiment Video
Updated: Mar 25, 2026
06:37
Author Spotlight: Modeling Retinal Pathologies with Enhanced RPE Cell-Based Disease Models
Published on: May 3, 2024
1.8K
Boosting foundation models for rare eye disease diagnosis via a multimodal text-to-image generative framework
Ruoyu Chen1, Weiyi Zhang1, Bowen Liu1
1School of Optometry, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China.
NPJ Digital Medicine
|March 24, 2026
Summary
EyeDiff, a new text-to-image model, generates realistic eye images to address data scarcity in training diagnostic AI. This improves diagnostic accuracy for common and rare retinal diseases.
Area of Science:
- Ophthalmology
- Artificial Intelligence
- Medical Imaging
Background:
- Vision-threatening retinal diseases are increasing globally.
- Deep learning (DL) models improve diagnostic efficiency but face data scarcity and imbalance issues, especially for rare diseases.
- Existing DL models struggle with limited and imbalanced datasets for training robust diagnostic tools.
Purpose of the Study:
- To introduce EyeDiff, a generative foundation model for synthesizing lesion-preserving ophthalmic images from text.
- To address data scarcity and imbalance in training diagnostic models for retinal diseases.
- To enhance the diagnostic accuracy of AI models for both common and rare eye conditions.
Main Methods:
- Developed EyeDiff, a text-to-image generative foundation model.
- Synthesized high-fidelity ophthalmic images from textual descriptions across multiple modalities.
- Augmented minority classes in 11 diverse, globally sourced datasets with generated images.
- Evaluated image quality using objective metrics and expert human assessments.
- Trained and tested various foundation models (modality-specific, multimodal, vision-language) with augmented data.
Main Results:
- EyeDiff generated high-fidelity images that accurately reflected textual disease descriptions.
- Generated images preserved crucial lesion details across various retinal diseases and imaging types.
- Data augmentation with EyeDiff consistently improved diagnostic accuracy for common and rare eye diseases.
- Enhanced performance was observed across different types of foundation models trained on augmented data.
Conclusions:
- EyeDiff offers a scalable solution for generating balanced, disease-relevant ophthalmic data.
- The model demonstrates potential as a general-purpose text-to-image tool for advancing retinal disease diagnosis.
- EyeDiff can overcome data limitations, improving the robustness and accuracy of AI diagnostic systems in ophthalmology.
