Mamba-based brain tumor segmentation of incomplete multi-modal MR images

Deep learning excels at multi-modal medical image segmentation, but its performance often drops with incomplete modalities, a frequent challenge in clinical settings. Accurate brain tumor segmentation from magnetic resonance imaging (MRI) scans is vital for diagnosis, treatment planning, and therapy assessment, yet, incomplete MRI data are common due to various clinical factors. The aim of this study is to improve brain tumor segmentation of incomplete MRI data.

To tackle this, we introduce a novel Mamba fusion (MF) network specifically designed to maintain segmentation performance even when MRI modalities are absent or incomplete. Our network utilizes multi-modal encoders to extract features from all available modalities. We have developed cross-level MF blocks that leverage a contextual learning mechanism to capture global features from low-level data. Additionally, a cross-level uncertainty (CU) constraint is applied to each class of the final predicted tumor, enhancing reliability.

Extensive experiments on the BraTS2018 and BraTS2020 datasets demonstrate that our method consistently outperforms existing state-of-the-art techniques across various incomplete multi-modal settings, and improves mean dice similarity coefficient (DSC) by about one point over the strongest baseline across whole tumor (WT)/tumor core (TC)/enhancing tumor (ET) on both datasets.

The proposed method allows to foster the learning of absent modality features, leading to a more comprehensive representation of multi-modal magnetic resonance (MR) images for tumor segmentation, mitigating the challenges associated with feature incompleteness due to absent modalities and enhancing the model's capability to navigate these complex situations.