Evaluating OCT Device-Reported Image Quality Score: Towards a Task-Specific Quality Gate for Deep Learning-based...

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Manufacturer-defined signal-strength indices are common quality benchmarks for automated optical coherence tomography (OCT) analysis.
• The predictive relationship between these traditional metrics and deep learning (DL) segmentation accuracy is not well-established.
• Existing metrics were developed for conventional image processing and may not be suitable for modern DL models.

Purpose of the Study:

• To empirically evaluate the Heidelberg Spectralis Q-score's ability to predict DL segmentation accuracy for posterior segment anatomical boundaries in OCT.
• To compare the Q-score's predictive power against standard metrics and the novel Earth Mover's Distance (EMD) for segmentation evaluation.
• To investigate the relationship between anatomical depth and segmentation error, and its potential confounding factors.

Main Methods:

• Evaluated the Heidelberg Spectralis Q-score against U-Net segmentation performance on 5,047 B-scans from 103 eyes.
• Assessed segmentation accuracy for the Ellipsoid Zone (EZ), Bruch's Membrane (BM), and Choroid Outer Boundary (COB).
• Utilized standard metrics (MAE, MSE, Dice) and adapted Earth Mover's Distance (EMD) for 2-D geometric boundary agreement.

Main Results:

• The Q-score explained less than 1.4% of the variance in DL segmentation accuracy across all three boundaries, indicating poor predictive ability.
• A consistent trend of increasing segmentation error with anatomical depth was observed (EZ < BM < COB), independent of signal strength.
• Paradoxical positive correlations at the COB were linked to increased choroidal thickness, mediated by higher Q-scores, highlighting localization challenges not captured by signal strength.

Conclusions:

• Signal-strength-based quality metrics like the Q-score are unreliable predictors of DL segmentation performance in OCT.
• Segmentation error increases with anatomical depth, a factor not accounted for by current signal-based quality assessments.
• A paradigm shift is needed towards task-specific acquisition quality criteria that are calibrated to DL model performance, not just signal interpretability.