A MultiRater MultiOrgan Abdominal CT Dataset for Calibration Analysis and Uncertainty Modeling in Segmentation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Biomedical Engineering

Background:

• Deep learning (DL) models face challenges in accurately segmenting ambiguous structures like tumors and organ boundaries in medical imaging.
• Inter-rater variability among experts leads to inconsistent segmentation outcomes, impacting clinical decision-making.
• Miscalibrated and overconfident DL models can mislead radiologists regarding malignancy risk assessment.

Purpose of the Study:

• To establish the CURVAS challenge (Calibration and Uncertainty for multiRater Volume Assessment in multiorgan Segmentation) to address segmentation uncertainty and calibration issues.
• To jointly assess uncertainty, calibration, and segmentation quality in multi-organ segmentation tasks.
• To promote clinical relevance by evaluating organ volumes while accounting for annotation variability.

Main Methods:

• A dataset of 90 contrast-enhanced CT scans from University Hospital Erlangen was curated.
• Pancreas, liver, and kidney segmentations were annotated by three independent experts.
• The CURVAS challenge focuses on developing and benchmarking algorithms for segmentation accuracy, calibration, and reliability.

Main Results:

• Quantitative analysis revealed strong consistency in kidney and liver segmentations.
• Pancreas segmentation proved challenging, highlighting the need for improved labeling protocols.
• The dataset serves as a foundation for developing robust segmentation algorithms.

Conclusions:

• Addressing uncertainty and calibration is crucial for reliable deep learning-based medical image segmentation.
• Standardized annotation practices and refined training strategies are necessary for challenging structures like the pancreas.
• The CURVAS dataset facilitates advancements in segmentation accuracy, calibration, and clinical applicability.