Domain-Shift AI Technology for Vendor-Agnostic Multiple Macular Disease Detection From 3D OCT Scans

A deep learning (DL) model capable of analyzing optical coherence tomography (OCT) 3-dimensional (3D) scans from various vendors is essential for robust disease detection.

To develop a vendor-agnostic model for multidisease classification using 3D scans from different vendors.

This multicenter retrospective cohort study included OCT scans from tertiary eye hospitals, a private eye center, an open online database, and retrospective and prospective research cohorts in Hong Kong and Vietnam. The model was trained exclusively on 3D scans from vendor 1 (Spectralis [Heidelberg Engineering]). Nine external datasets, including 3D scans from vendor 1, 3D scans from vendor 2 (Cirrus [Zeiss]), and 2-dimensional (2D) scans from vendor 1, were used for testing. A category labeled uncertain was introduced to handle previously unseen macular conditions. A triage module was incorporated into the model. 3D scans from vendor 1 were retrospectively collected from the period January 2010 through September 2022, and external data sets (from vendors 1 and 2) were retrospectively collected from January 2008 through September 2022. Model development and data analysis were conducted between July 2022 and September 2024.

A Residual Neural Network (ResNet) 3D model was trained using 2 different architectures for comparison. An unsupervised test time domain adaptation method, Test Entropy, was used to address vendor-to-vendor domain discrepancies.

The primary outcomes were area under the receiver operating characteristic curve (AUROC), micro-average positive predictive value (PPV), micro-average negative predictive value (NPV), and clinically important miss rate.

A total of 6756 OCT scans from 1669 patients were used for model development (mean [SD] age, 70.6 [16.5] years; 742 female patients [44.5%]), and 12 236 OCT scans from 4336 patients were used for external testing (mean [SD] age, 63.9 [10.4] years; 2099 female [51.2%]). AUROC ranged from 0.779 (95% CI, 0.752-0.806) to 0.999 (95% CI, 0.995-1.000) for vendor 1, from 0.754 (95% CI, 0.615-0.872) to 0.991 (95% CI, 0.980-0.999) for vendor 2, and from 0.801 (95% CI, 0.781-0.821) to 0.950 (95% CI, 0.943-0.956) for 2D scans. Micro-average PPVs ranged from 56.7% (95% CI, 54.5%-58.7%) to 72.0% (95% CI, 68.9%-74.9%) for vendor 1 and from 46.0% (95% CI, 43.6%-48.4%) to 60.4% (95% CI, 58.3%-62.4%) for vendor 2. All micro-average NPVs exceeded 97.5% (95% CI, 97.1%-97.8%). The uncertain category demonstrated high specificity (>95.0%; 95% CI, 93.4%-96.2%) and accuracy (>92.7%; 95% CI, 90.9%-94.1%) but varied sensitivity (from 14.3%; 95% CI, 2.6%-51.3%; to 83.3%; 95% CI, 60.8%-94.2%). The clinically important miss rates were 6.16% (45 of 731) and 6.70% (13 of 194) for urgent cases and 4.41% (47 of 1066) and 8.67% (50 of 577) for semi-urgent cases for vendors 1 and 2, respectively.

The results of this multicenter cohort study highlight the potential of this vendor-agnostic DL model for broad deployment across diverse eye care settings, streamlining detection and triage in modern ophthalmic practices.