Related Experiment Videos
GENERALIZABILITY OF DEEP LEARNING MODELS FOR REFERABLE DIABETIC RETINOPATHY DETECTION: A CROSS-POPULATION STUDY
Ceren Durmaz Engin1,2, Mustafa Alper Selver3, Seher Köksaldı4
1Department of Ophthalmology, Izmir Democracy University Buca Seyfi Demirsoy Education and Research Hospital, Izmir, Turkey.
Purpose:
To evaluate cross-population performance of deep learning models for referable diabetic retinopathy (DR) detection and assess whether inclusion of local data improves robustness in a Turkish external validation setting.
Methods:
Global datasets (DDR, IDRiD, Messidor) and a Turkish real-world clinical dataset were used. Images were re-graded by three ophthalmologists and binarized as referable/non-referable DR. Five architectures were evaluated: EfficientNet-B0, EfficientNet-V2, ResNet50, ConvNeXt-V2-Tiny, and Swin V2 CR Small. Models were tested under three settings: Global Train-Global Test, Global Train-Local Test, and Mixed Train-Mixed Test. Performance metrics included accuracy, sensitivity, specificity, NPV, ROC-AUC, and PR-AUC; Grad-CAM was used for explainability.
Results:
The global training set included 9,881 images; two external test sets (2,042 images each) were used. Inter-grader agreement was high (ICC[3,1]=0.84; ICC[3,k]=0.96). Swin V2 CR Small achieved the highest performance in Global Train-Global Test (accuracy 0.938; ROC-AUC 0.985). Performance declined in Global Train-Local Test, where ConvNeXt-V2-Tiny performed best (accuracy 0.823; sensitivity 0.940; ROC-AUC 0.913). In Mixed Train-Mixed Test, ConvNeXt-V2-Tiny remained the most balanced model (accuracy 0.823; sensitivity 0.939; NPV 0.948; ROC-AUC 0.875).
Conclusion:
Models performed strongly under matched conditions but degraded across populations. ConvNeXt-V2-Tiny showed the most stable screening performance, supporting the need for local validation before deployment.
