Uncertainty-aware auto-segmentation enables adaptive radiotherapy planning of liver metastases

Despite remarkable auto-segmentation performance and growing clinical adoption, models frequently fail to generalize beyond the training distribution. Therefore, this study aimed to develop and validate an uncertainty-aware segmentation pipeline that yields high delineation accuracy for liver cancer patients and reliably identifies safety-critical segmentation imperfections in radiotherapy.

Datasets of n = 173 patients with 224 liver metastases were divided into training (n = 100), validation (n = 33), and test (n = 40). Each case included annotated T2-weighted magnetic resonance images. An nnU-Net segmented treatment-relevant organs and provided a margin-extended hepatic mask, which served as input to a second-stage SE-ResUNet for lesion delineation. Uncertainty was approximated via checkpoint ensembling under a cyclic learning-rate schedule, yielding entropy-based uncertainty maps and case-wise uncertainty scores. These guided an adaptive post-inference analysis consisting of (i) detection of potentially missed lesions and (ii) identification of problematic contours. Performance was evaluated on 40 patients (n = 52 lesions) using Dice Similarity Coefficient (DSC), 95%-Hausdorff-Distance (HD95), expected calibration error (ECE), uncertainty coverage, and the correlation between case-wise uncertainty scores and segmentation accuracy.

Overall performance across treatment-relevant organs was DSC/HD95 = 0.89/4.8 mm; lesionsegmentation achieved DSC/HD95 = 0.86/4.9 mm. Uncertainty-guided post-processing, triggered in 14/40 cases, recovered 7 initially missed metastases, increasing lesion-level sensitivity from 84.6% (44/52) to 98.1% (51/52). Uncertainty estimates showed good calibration with ECE/uncertainty coverage for organs/lesion of 7.2/93.7% and 7.5/98.3%, and significant correlation between uncertainty/DSC (r = -0.802).

The proposed pipeline seamlessly integrates accurate organ/lesion segmentation, interpretable uncertainty, and anatomy-guided post-processing to detect safety-critical segmentation failures and enable trustworthy AI auto-segmentation in clinical routine.