Uncertainty-aware multiple-instance learning for reliable classification: Application to optical coherence tomography | JoVE Visualize

Area of Science:

Medical image analysis
Artificial intelligence
Computer vision

Background:

Deep learning models for medical imaging often exhibit reduced performance on data from scanners different from those used for training.
Vendor-specific artifacts in medical scans are a primary cause of this poor generalizability.
Existing models may not reliably perform across diverse clinical settings without retraining.

Purpose of the Study:

To introduce and evaluate Uncertainty-Based Instance eXclusion (UBIX), a novel method to enhance the reliability of deep learning classification models.
To improve the generalizability of medical image analysis models to data from different vendors and scanners.
To address the challenge of vendor-specific artifacts impacting model performance in optical coherence tomography (OCT) for age-related macular degeneration staging.

Main Methods:

UBIX is an inference-time module designed for multiple-instance learning (MIL) settings.
It detects and reduces the contribution of artifact-corrupted instances (e.g., 2D slices) to the overall bag-level prediction (e.g., volumetric image) using uncertainty estimation.
Out-of-distribution (OOD) detection is employed to identify instances with unseen artifacts.

Main Results:

When applied to external datasets with vendor-specific artifacts, UBIX demonstrated reliable performance, with a slight decrease in quadratic weighted kappa (κw) from 0.861 to 0.708.
A state-of-the-art 3D neural network without UBIX experienced a significant performance drop (κw from 0.852 to 0.084) on the same external datasets.
UBIX successfully identified instances with unseen artifacts, reducing their influence on predictions without requiring model retraining.

Conclusions:

UBIX effectively enhances the reliability and generalizability of deep learning models in medical image analysis, particularly in the presence of scanner-specific artifacts.
The method improves model robustness when applied to data from different vendors, mitigating performance degradation.
UBIX offers a promising solution for increasing the practical applicability of artificial intelligence in diverse clinical environments without the need for extensive retraining.