Automatic correction of performance drift under acquisition shift in medical image classification.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Machine Learning

Background:

• Image-based disease detection models are vulnerable to performance drift caused by changes in data acquisition (e.g., new hardware, software updates).
• This drift can impact clinical decision-making, even if models maintain generalizability metrics like area under the receiver-operating characteristic curve.
• Maintaining consistent model performance is crucial for safe and effective clinical deployment.

Purpose of the Study:

• To propose a novel, generic, and automatic recalibration method for unsupervised prediction alignment.
• To address the challenge of performance drift in image-based disease detection models without requiring ground truth annotations.
• To demonstrate the method's effectiveness in real-world medical imaging scenarios.

Main Methods:

• Developed Unsupervised Prediction Alignment (UPA), a method requiring only limited unlabeled data from the shifted distribution.
• Applied UPA to detect and correct performance drift in mammography-based breast cancer screening.
• Validated the method on publicly available histopathology datasets.

Main Results:

• UPA effectively detected and corrected performance drifts caused by realistic image acquisition shifts.
• The method preserved expected performance in terms of sensitivity and specificity across various scenarios.
• Demonstrated robustness in both mammography and histopathology image analysis.

Conclusions:

• Unsupervised Prediction Alignment offers a vital safeguard for the clinical deployment of image-based diagnostic models.
• The method ensures reliable performance by mitigating the impact of data acquisition variability.
• UPA provides a practical solution for maintaining model accuracy in dynamic clinical environments.