MSCT-UNET: multi-scale contrastive transformer within U-shaped network for medical image segmentation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Automatic multi-organ segmentation is crucial for medical diagnosis and treatment planning.
• U-shaped neural networks (UNets) with encoder-decoder architectures excel in segmentation but struggle with long-range dependencies (CNNs) or pixel-level details (transformers).

Purpose of the Study:

• To develop a hybrid network, MSCT-UNET, that overcomes the limitations of pure CNNs and transformers for multi-organ segmentation.
• To improve segmentation performance by fusing multi-scale CNN and transformer features and incorporating multi-task contrastive learning.

Main Methods:

• Proposed MSCT-UNET, a hybrid network fusing CNN and transformer features at multiple scales.
• Implemented a cross-fusion block for deep integration of low-level and high-level features.
• Introduced multi-task contrastive learning (global self-supervised and local supervised) to enhance network representation.
• Utilized a transformer in the decoder to refine segmentation maps.

Main Results:

• MSCT-UNET demonstrated superior performance on ACDC, Synapase, and BraTS datasets compared to existing methods.
• Ablation studies confirmed the effectiveness of the proposed hybrid architecture and multi-task contrastive learning.
• The network successfully captured multi-scale long-range dependencies and fine-grained details.

Conclusions:

• MSCT-UNET effectively integrates global context and local details for accurate multi-organ segmentation.
• The hybrid approach and contrastive learning significantly enhance segmentation capabilities.
• This method offers a promising advancement for medical image analysis in clinical applications.