Prototype bank-driven test-time adaptation for medical ultrasound image segmentation

Deep learning has achieved remarkable success in medical image segmentation, particularly in ultrasound imaging, where deep neural networks have demonstrated near-expert performance. However, these models typically assume that training and test data follow the same distribution-an assumption that often fails in real-world clinical practice due to domain shifts caused by variations in imaging devices, acquisition protocols, and operator techniques. These discrepancies can significantly degrade model performance. Existing solutions-such as supervised fine-tuning, unsupervised domain adaptation, and domain generalization-require either costly labeled data or access to source domain data, limiting their scalability and clinical applicability.

To address domain shift in real-world ultrasound image segmentation, this study proposes a test-time adaptation (TTA) framework that eliminates the need for source data or target labels, while ensuring robustness against distributional drift and catastrophic forgetting.

We present Prototype Bank-Driven Test-Time Adaptation (PBTTA), a novel TTA framework consisting of two key modules: (1) the Dynamic Statistics Fusion Module (DSFM), which enables domain-level adaptation by dynamically adjusting batch normalization layers using fused statistics from the test sample and source domain; and (2) the Prototype Bank-Guided Semantic Adaptation Module (PBSAM), which maintains a dynamic prototype bank for each semantic class to support semantic-level adaptation. PBTTA employs a dual-classifier strategy that combines a prototype-based classifier for stable semantic guidance and a parameter-based classifier for flexible decision-making. Notably, PBTTA does not require backpropagation to update model parameters during test time adaptation phase, ensuring efficient and stable adaptation.

PBTTA achieves state-of-the-art performance across both ultrasound breast and thyroid tumor segmentation tasks. On average, it improves the Dice score by 15.04% (to 64.82%) for breast tumor segmentation and by 8.88% (to 57.45%) for thyroid tumor segmentation, compared to non-adaptive baselines. Moreover, PBTTA exhibits excellent robustness under continuous domain shifts and effectively mitigates catastrophic forgetting.