Multi-modal semi-supervised medical image segmentation via spatial weight fusion and prototype-based alignment.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multi-modal learning enhances medical image segmentation by integrating complementary data from different sources.
• Existing methods struggle with the scarcity of high-quality annotations and anatomical misalignment between modalities in clinical settings.

Purpose of the Study:

• To introduce MHPC-Net, a novel semi-supervised network for multi-modal medical image segmentation.
• To address challenges of limited annotations and anatomical misalignment in cross-modal segmentation.

Main Methods:

• MHPC-Net employs a dual-branch architecture for CT and MRI integration, featuring a Manhattan Hybrid-attentive Prototype-aligned Cross-modal Network.
• A feature interaction module uses Manhattan distance and cross-attention for enhanced inter-modal exchange and detail preservation.
• Modality contrast strategy and prototype alignment with a memory bank ensure semantic consistency and robust representation learning.

Main Results:

• MHPC-Net achieves state-of-the-art performance in semi-supervised multi-modal segmentation tasks with limited labels.
• The network demonstrates improved accuracy and generalization capabilities in cardiac and abdominal segmentation experiments.
• Key innovations include effective handling of anatomical misalignment and semantic inconsistency.

Conclusions:

• MHPC-Net offers a robust solution for semi-supervised multi-modal medical image segmentation under data scarcity.
• The proposed methods effectively mitigate modality misalignment and enhance cross-modal representation learning.
• This work advances the field by improving segmentation accuracy and generalization in challenging clinical scenarios.