Customizing AI-based screening with real-world data: Practical insights from diabetic retinopathy

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Diabetic retinopathy (DR) is a primary cause of vision loss globally in adults with diabetes.
• Existing AI screening tools (IDx-DR, RetCAD) demonstrate high sensitivity in controlled settings.
• Real-world DR screening faces challenges with image quality and local healthcare adaptation.

Purpose of the Study:

• To compare AI algorithms (IDx-DR, RetCAD) for non-mydriatic DR screening against ophthalmologist mydriatic fundoscopy.
• To evaluate the impact of customized referral threshold modification ('Greifswald modification') on AI screening outcomes.
• To assess AI performance considering image quality and patient inclusion in real-world settings.

Main Methods:

• A one-centre observational study included 1716 patients with diabetes mellitus.
• Assessed sensitivity, specificity, ungradable image proportion, and reduction in ophthalmologic evaluations.
• Customized referral thresholds using the Youden Index for regression-based AI algorithms.

Main Results:

• High rates of ungradable images were observed (5.7% unacquired, 2.1% incomplete for IDx-DR).
• AI screening reduced ophthalmologic exam needs by 47.5% to 78.5%.
• Sensitivity varied: 70.4% (RetCAD) to 93.6% (RetCAD with Greifswald modification) for analysable images; 52.7% (IDx-DR) to 79.9% (RetCAD with Greifswald modification) including all patients.

Conclusions:

• Real-world AI DR screening performance can be lower than in controlled studies when including non-analysable patients.
• Regression AI algorithms allow referral threshold customization, enhancing screening accuracy.
• AI screening effectively reduces the clinical burden of DR evaluations in diabetes care.