MI²-Net: A Mamba-based network for joint incomplete multi-modal and incomplete label MRI image segmentation

Area of Science:

• Medical Image Analysis
• Artificial Intelligence in Medicine
• Computational Imaging

Background:

• Clinical neuroimaging often encounters incomplete data modalities and sparse annotations for segmentation tasks.
• Existing multi-modal segmentation methods struggle with simultaneous data deficiencies, limiting practical application.
• Addressing these dual challenges in isolation overlooks their compound impact on segmentation performance.

Purpose of the Study:

• To propose the first framework, MI²-Net, specifically designed for multi-modal MRI segmentation under dual-missing conditions (incomplete modalities and sparse labels).
• To develop novel modules for handling missing data and leveraging limited annotations effectively.
• To advance the state-of-the-art in robust medical image segmentation for clinical practice.

Main Methods:

• Introduced a missing-aware mamba imputation & fusion module with a tri-planar mamba encoder and multi-scale imputation blocks for reconstructing semantic features.
• Implemented a hybrid mamba fusion module to enhance inter-modal feature interactions.
• Employed a fusion-based semi-supervised learning strategy with a dual-decoder architecture and a specific auxiliary regularizer for knowledge propagation and representation enhancement.

Main Results:

• MI²-Net demonstrated superior performance compared to existing missing-modality and semi-supervised segmentation methods across three public datasets.
• The framework achieved a significant Dice Similarity Coefficient (DSC) gain exceeding 0.90% on BraTS2018, even with fewer labeled cases than traditional methods.
• The proposed approach effectively handles the more challenging dual-missing scenario in multi-modal MRI segmentation.

Conclusions:

• MI²-Net is the first effective framework for multi-modal MRI segmentation addressing both incomplete modalities and sparse annotations simultaneously.
• The proposed imputation and fusion modules, combined with semi-supervised learning, significantly improve segmentation robustness and accuracy.
• This work offers a promising solution for real-world clinical scenarios where data limitations are common.