MI²-Net: A Mamba-based network for joint incomplete multi-modal and incomplete label MRI image segmentation

Area of Science:

• Medical Image Analysis
• Artificial Intelligence in Medicine
• Machine Learning for Healthcare

Background:

• Current multi-modal segmentation methods require complete data and labels, which are often unavailable in clinical practice.
• Existing approaches address missing modalities and sparse annotations separately, failing to account for their combined impact.
• This limitation hinders the practical application of advanced segmentation techniques in real-world medical settings.

Purpose of the Study:

• To propose the first framework, MI²-Net, specifically designed for multi-modal MRI segmentation under dual-missing conditions (incomplete modalities and sparse annotations).
• To develop novel modules for imputing missing data and fusing information from available modalities.
• To implement a semi-supervised learning strategy to leverage unlabeled data effectively, overcoming annotation scarcity.

Main Methods:

• Introduced a missing-aware mamba imputation & fusion module utilizing parallel tri-planar global-local mamba encoders and multi-scale mamba imputation blocks for robust feature reconstruction and inter-modal fusion.
• Employed a fusion-based semi-supervised learning strategy with a dual-decoder architecture and mutual consistency regularization to propagate knowledge from labeled to unlabeled data.
• Designed a specific semi-supervised auxiliary regularizer using high-quality soft pseudo-labels to enhance modality-specific decoding.

Main Results:

• MI²-Net demonstrated superior performance compared to existing missing-modality and semi-supervised segmentation methods across three public datasets.
• In the challenging BraTS2018 dataset, MI²-Net achieved a significant Dice Similarity Coefficient (DSC) gain exceeding 0.90% despite using fewer labeled cases than baseline methods.
• The framework effectively addresses the dual-missing scenario, outperforming methods trained on complete data.

Conclusions:

• MI²-Net is the first effective framework for multi-modal MRI segmentation in the dual-missing setting, addressing both incomplete modalities and sparse annotations.
• The proposed imputation and fusion modules, combined with semi-supervised learning, significantly enhance segmentation accuracy and robustness.
• MI²-Net offers a practical solution for real-world clinical applications where data deficiencies are common.