Related Experiment Videos
Prospective Data Curation Enables High-Performance Artificial Intelligence for Diabetic Retinopathy Screening in a
Cameron M Ashrafzadeh1, Milan Bahi1, Amira Mostafa2
1Beetham Eye Institute, Joslin Diabetes Center, Boston, Massachusetts.
Purpose:
To determine whether a high-quality, prospectively curated dataset can, by itself, enable the development of robust and clinically effective artificial intelligence as a medical device (AIaMD) models for diabetic retinopathy (DR) screening, even with minimal artificial intelligence (AI) infrastructure. This study evaluates whether careful data curation, standardized acquisition, and rigorous grading processes can yield high-performing AI models for more-than-mild DR (MTM) and diabetic macular edema (DME) detection from ultra-widefield (UWF) fundus images.
Design:
An evaluation of diagnostic technology.
Subjects:
Patients with diabetes receiving imaging at Alexandria iCare Retina Reading Center.
Methods:
A total of 152 025 UWF color images were collected between February 2022 and June 2024 at a UWF image reading center during routine screening. This was a cross-sectional study with patient consent obtained at the time of imaging. After excluding noncolor or ungradable images, 26 232 UWF color images from 5394 diabetic patients were used to train 2 Inception V3-based convolutional neural networks: one to detect MTM and another for OCT-confirmed DME. Images were split 80:10:10 by patient into training, validation, and test sets. Models were trained on red-green channel inputs using standard augmentation and Adam optimization (learning rate 0.0005). Prospective validation was performed on 12 698 additional images from 3096 patients collected between July and December 2024, following identical imaging and adjudication protocols.
Main Outcome Measures:
Area under the curve (AUC), sensitivity, and specificity.
Results:
In baseline testing, the MTM model achieved an AUC of 0.962 ± 0.003, sensitivity of 0.922 ± 0.001, and specificity of 0.873 ± 0.010; the DME model achieved an AUC of 0.879 ± 0.014, sensitivity of 0.809 ± 0.022, and specificity of 0.790 ± 0.023. In the prospective dataset, the MTM model maintained strong performance (AUC 0.949; sensitivity 0.86-0.89; specificity 0.85-0.89), while the DME model yielded an AUC of 0.821 with balanced sensitivity (0.76) and specificity (0.73). Gradient-weighted class activation mapping visualizations confirmed focus on clinically relevant lesions.
Conclusions:
This study demonstrates that rigorous prospective data collection and quality control can produce high-performing AIaMDs even with limited AI engineering resources. Locally curated datasets aligned with regional populations, equipment, and workflows can yield reliable, regulation-ready tools that advance equitable DR screening in resource-limited settings.
Financial Disclosures:
Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
