Consistency-guided Uncertainty Estimation for Source-Free Medical Image Segmentation

PPseudo-label self-training is a cornerstone of Source-Free Domain Adaptation (SFDA) in medical imaging, valued for improving performance while protecting data privacy. However, its effectiveness is often limited by two flawed assumptions: that source models are trained on perfect, noise-free labels, and that standard uncertainty measures can accurately catch pseudo-label errors. In practice, medical datasets frequently contain structural noise, and typical estimators often miss high-confidence mistakes like over-segmentation or anatomically impossible shapes. To address these challenges, we propose a novel two-stage SFDA framework that enhances pseudo-label quality through consistency-guided uncertainty estimation and structure-aware refinement. First, we use aggressively augmented and mixed views of target images to estimate prediction stability via Chebyshev's inequality, automatically partitioning the target domain into high- and low-confidence subsets. Second, we apply tailored strategies for each subset: high-confidence images undergo instance-specific style normalization and are refined using affinity graphs with random walk propagation to improve structural coherence, while low-confidence images are regularized via multi-view consistency to encourage prediction stability. Extensive experiments on multiple datasets spanning three imaging modalities (color fundus photography, endoscopy, and MRI) demonstrate the superiority of our method over state-of-the-art approaches, even under significant domain shift and label noise. Our approach exhibits strong robustness and generalizability, making it well-suited for real-world clinical applications where annotation quality is variable. Our code is available at https://github.com/iMED-Lab/CUE-SFDA.