Related Experiment Video
Updated: Aug 1, 2025
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
Development and international validation of custom-engineered and code-free deep-learning models for detection of
Siegfried K Wagner1, Bart Liefers2, Meera Radia3
1NIHR Moorfields Biomedical Research Centre, London, UK; Institute of Ophthalmology, University College London, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK.
Insights
Code-free deep learning models show promise for diagnosing retinopathy of prematurity (ROP) plus disease, aiding early detection and preventing childhood blindness. These AI tools offer a sustainable solution for ROP screening, especially in underserved regions.
Area of Science:
- Ophthalmology
- Artificial Intelligence
- Medical Imaging
Background:
- Retinopathy of prematurity (ROP) is a leading cause of childhood blindness, necessitating expert diagnosis via interval screening.
- Increasing survival rates of premature neonates and a shortage of pediatric ophthalmologists challenge the sustainability of current ROP screening methods.
- Code-free deep learning (CFDL) offers a potential solution, reducing reliance on expert data scientists and benefiting low-resource healthcare settings.
Purpose of the Study:
- To develop and validate bespoke and code-free deep learning (CFDL) classifiers for detecting plus disease, a key indicator of ROP.
- To assess the performance of these models in diverse ethnic, geographic, and socioeconomic populations.
- To evaluate the potential of CFDL for ROP screening and preventing blindness in vulnerable neonates.
Main Methods:
- A retrospective cohort study involving 1370 neonates with retinal images acquired between 2008 and 2018.
- Development of bespoke and CFDL models for discriminating healthy, pre-plus, and plus disease stages of ROP.
- Internal validation on 200 images and external validation on 338 images from four international datasets using different imaging devices.
Main Results:
- Both bespoke and CFDL models achieved high performance in discriminating healthy from pre-plus/plus disease internally (AUCs 0.986 and 0.989, respectively).
- Models demonstrated good generalization to external datasets acquired with the same imaging device (Retcam).
- Performance decreased when discriminating minority classes (pre-plus disease) or when using a different imaging device (3nethra neo).
Conclusions:
- Bespoke and CFDL models show comparable performance to senior ophthalmologists in identifying ROP plus disease features.
- CFDL models may have limitations in generalizing to minority classes and require careful consideration of imaging device variations.
- Further validation is warranted, supporting the potential role of code-free AI approaches in ROP screening to prevent childhood blindness.
Background:
Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed through interval screening by paediatric ophthalmologists. However, improved survival of premature neonates coupled with a scarcity of available experts has raised concerns about the sustainability of this approach. We aimed to develop bespoke and code-free deep learning-based classifiers for plus disease, a hallmark of ROP, in an ethnically diverse population in London, UK, and externally validate them in ethnically, geographically, and socioeconomically diverse populations in four countries and three continents. Code-free deep learning is not reliant on the availability of expertly trained data scientists, thus being of particular potential benefit for low resource health-care settings.
Methods:
This retrospective cohort study used retinal images from 1370 neonates admitted to a neonatal unit at Homerton University Hospital NHS Foundation Trust, London, UK, between 2008 and 2018. Images were acquired using a Retcam Version 2 device (Natus Medical, Pleasanton, CA, USA) on all babies who were either born at less than 32 weeks gestational age or had a birthweight of less than 1501 g. Each images was graded by two junior ophthalmologists with disagreements adjudicated by a senior paediatric ophthalmologist. Bespoke and code-free deep learning models (CFDL) were developed for the discrimination of healthy, pre-plus disease, and plus disease. Performance was assessed internally on 200 images with the majority vote of three senior paediatric ophthalmologists as the reference standard. External validation was on 338 retinal images from four separate datasets from the USA, Brazil, and Egypt with images derived from Retcam and the 3nethra neo device (Forus Health, Bengaluru, India).
Findings:
Of the 7414 retinal images in the original dataset, 6141 images were used in the final development dataset. For the discrimination of healthy versus pre-plus or plus disease, the bespoke model had an area under the curve (AUC) of 0·986 (95% CI 0·973-0·996) and the CFDL model had an AUC of 0·989 (0·979-0·997) on the internal test set. Both models generalised well to external validation test sets acquired using the Retcam for discriminating healthy from pre-plus or plus disease (bespoke range was 0·975-1·000 and CFDL range was 0·969-0·995). The CFDL model was inferior to the bespoke model on discriminating pre-plus disease from healthy or plus disease in the USA dataset (CFDL 0·808 [95% CI 0·671-0·909, bespoke 0·942 [0·892-0·982]], p=0·0070). Performance also reduced when tested on the 3nethra neo imaging device (CFDL 0·865 [0·742-0·965] and bespoke 0·891 [0·783-0·977]).
Interpretation:
Both bespoke and CFDL models conferred similar performance to senior paediatric ophthalmologists for discriminating healthy retinal images from ones with features of pre-plus or plus disease; however, CFDL models might generalise less well when considering minority classes. Care should be taken when testing on data acquired using alternative imaging devices from that used for the development dataset. Our study justifies further validation of plus disease classifiers in ROP screening and supports a potential role for code-free approaches to help prevent blindness in vulnerable neonates.
Funding:
National Institute for Health Research Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and the University College London Institute of Ophthalmology.
Translations:
For the Portuguese and Arabic translations of the abstract see Supplementary Materials section.
