Mamba-Convolutional UNet for multi-modal medical image synthesis.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multi-modal medical imaging is crucial for preoperative evaluation but faces challenges in data acquisition and cost.
• Acquiring sufficient paired multi-modal data for training cross-modality synthesis models is logistically difficult and expensive.

Purpose of the Study:

• To develop a novel dual-branch architecture, Mamba-Convolutional UNet, for efficient multi-modal medical image synthesis.
• To enable effective cross-modal synthesis even with limited paired training data by introducing a reprogramming layer.

Main Methods:

• The Mamba-Convolutional UNet employs a U-shaped architecture with parallel Mamba (SSM) and convolutional branches.
• Mamba captures long-range dependencies, while convolutions extract local features; an attention mechanism integrates these.
• A reprogramming layer in the Mamba module facilitates knowledge transfer for cross-modal synthesis with scarce data.

Main Results:

• Mamba-Convolutional UNet significantly outperformed six baseline models across five synthesis tasks and three datasets.
• The model achieved performance comparable to state-of-the-art methods with only 25% of the data for fine-tuning on new tasks.

Conclusions:

• The dual-branch Mamba-Convolutional UNet effectively integrates global and local features for superior medical image synthesis.
• The reprogramming layer in Mamba addresses target modality transformation challenges under data-limited conditions.