GMM-PA: Gaussian Mixture Model-Based Prototype Alignment for Multi-source Domain Adaptation in Polyp Segmentation

Accurate polyp segmentation in colonoscopy images is essential for the early detection and treatment of colorectal cancer. However, deep learning-based segmentation models often suffer performance degradation on data from unseen centers due to domain shift. To address this challenge, we propose a multi-source domain adaptation network based on Gaussian Mixture Model-guided Prototype Alignment (GMM-PA) for robust cross-domain polyp segmentation. First, an image preprocessing module is designed to visually mitigate domain discrepancies by applying color transformation in the LAB space and low-frequency spectral exchange via Fourier transform. Then, a hybrid CNN-Mamba architecture is introduced for feature extraction, combining the local modeling capacity of CNNs with the global context modeling of Mamba, offering lower computational cost compared to CNN-Transformer models. To better capture complex multimodal feature distributions, Gaussian mixture models are used to derive reliable class prototypes, addressing the limitations of previous unimodal assumptions. A cross-domain prototype feature refinement module further enhances representation quality by leveraging semantic relationships between features and prototypes. Additionally, an adversarial learning strategy is adopted to promote domain-invariant feature learning during training. Extensive experiments on three public polyp segmentation datasets demonstrate that the proposed GMM-PA method achieves superior performance compared to existing state-of-the-art approaches in cross-domain settings, achieving average Dice scores of 0.8396 and mIoU of 0.8449 across the datasets.