Average Calibration Losses for Reliable Uncertainty in Medical Image Segmentation

Area of Science:

• Medical Image Analysis
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) in medical imaging often exhibit overconfidence, reducing their reliability and clinical applicability.
• Calibration, the alignment of predicted confidence with true accuracy, is crucial for trustworthy AI in healthcare.

Purpose of the Study:

• To develop and evaluate differentiable formulations of marginal L1 Average Calibration Error (mL1-ACE) as an auxiliary loss function.
• To improve pixel-wise calibration in deep neural networks for medical image segmentation.
• To provide a method for controlling the trade-off between model calibration and segmentation accuracy.

Main Methods:

• Proposed differentiable mL1-ACE as a per-image auxiliary loss.
• Compared hard-binning and soft-binning approaches for pixel-wise calibration.
• Evaluated performance on ACDC, AMOS, KiTS, and BraTS datasets.
• Introduced dataset reliability histograms for analyzing calibration variability.

Main Results:

• mL1-ACE significantly reduced Average Calibration Error (ACE) and Maximum Calibration Error (MCE) across datasets.
• Hard-binned mL1-ACE maintained high Dice Similarity Coefficients (DSCs) while improving calibration.
• Soft-binned mL1-ACE showed greater calibration improvements but sometimes compromised segmentation performance.
• Dataset reliability histograms demonstrated better alignment between predicted confidences and true accuracies.

Conclusions:

• Differentiable mL1-ACE effectively enhances the calibration of DNNs for medical image segmentation.
• The choice between hard- and soft-binning allows for balancing calibration gains and segmentation performance.
• The proposed methods offer practitioners better control over reliability, facilitating clinical adoption of AI tools.