Related Experiment Video
Updated: Jul 28, 2026
Oxygen-Induced Retinopathy Model for Ischemic Retinal Diseases in Rodents
Published on: September 16, 2020
Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP
Amelia Bai1,2,3, Shuan Dai1,3,4, Jacky Hung2
1Department of Ophthalmology, Queensland Children's Hospital, South Brisbane, Queensland, Australia.
Insights
An artificial intelligence (AI) tool, ROP.AI, demonstrated effectiveness in detecting plus disease, a severe form of retinopathy of prematurity (ROP), in a multicenter Australian study. This AI shows promise for improving ROP screening and diagnosis in premature infants.
Area of Science:
- Ophthalmology
- Medical Imaging
- Artificial Intelligence
Background:
- Retinopathy of prematurity (ROP) is a significant cause of vision loss in premature infants.
- Detecting plus disease, a severe ROP indicator, is challenging due to subjective and time-intensive retinal examinations.
- Existing artificial intelligence (AI) algorithms for ROP detection need validation in diverse populations.
Purpose of the Study:
- To validate ROP.AI, an AI algorithm trained on a single cohort, for detecting plus disease in a multicenter Australian cohort.
- To assess the performance of ROP.AI in a real-world clinical setting across multiple tertiary centers.
- To determine the generalizability of ROP.AI for identifying severe retinopathy of prematurity.
Main Methods:
- Retinal images from routine ROP screening (May 2021-February 2022) across five Australian centers were collected.
- Images were analyzed using ROP.AI, and its diagnostic output was compared against expert ophthalmologist diagnoses.
- Performance metrics including sensitivity, specificity, and area under the receiver operator curve (AUC) were calculated.
Main Results:
- A total of 8052 images were analyzed.
- ROP.AI achieved an AUC of 0.75 for diagnosing plus disease.
- Optimized operating point yielded 84% sensitivity, 43% specificity, and 96% negative predictive value for plus disease detection.
Conclusions:
- ROP.AI successfully detected plus disease in an external, multicenter cohort, validating its performance beyond its initial training data.
- The algorithm demonstrated applicability in real-world conditions without image preprocessing or augmentation.
- Further training could enhance ROP.AI's generalizability for widespread clinical implementation in ROP screening.
Purpose:
Retinopathy of prematurity (ROP) is a sight-threatening vasoproliferative retinal disease affecting premature infants. The detection of plus disease, a severe form of ROP requiring treatment, remains challenging owing to subjectivity, frequency, and time intensity of retinal examinations. Recent artificial intelligence (AI) algorithms developed to detect plus disease aims to alleviate these challenges; however, they have not been tested against a diverse neonatal population. Our study aims to validate ROP.AI, an AI algorithm developed from a single cohort, against a multicenter Australian cohort to determine its performance in detecting plus disease.
Methods:
Retinal images captured during routine ROP screening from May 2021 to February 2022 across five major tertiary centers throughout Australia were collected and uploaded to ROP.AI. AI diagnostic output was compared with one of five ROP experts. Sensitivity, specificity, negative predictive value, and area under the receiver operator curve were determined.
Results:
We collected 8052 images. The area under the receiver operator curve for the diagnosis of plus disease was 0.75. ROP.AI achieved 84% sensitivity, 43% specificity, and 96% negative predictive value for the detection of plus disease after operating point optimization.
Conclusions:
ROP.AI was able to detect plus disease in an external, multicenter cohort despite being trained from a single center. Algorithm performance was demonstrated without preprocessing or augmentation, simulating real-world clinical applicability. Further training may improve generalizability for clinical implementation.
Translational Relevance:
These results demonstrate ROP.AI's potential as a screening tool for the detection of plus disease in future clinical practice and provides a solution to overcome current diagnostic challenges.
