MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images.

Area of Science:

• Computer Vision
• Remote Sensing
• Geospatial Analysis

Background:

• Semantic segmentation of remote sensing images is crucial for land cover, environmental, and urban planning.
• Multi-modal fusion models outperform single-modal ones but often struggle with remote modeling or computational cost.
• Existing Convolutional Neural Network (CNN) and Vision Transformer (ViT) based fusion methods have limitations.

Purpose of the Study:

• Introduce MFMamba, a novel Mamba-based multi-modal fusion network for remote sensing semantic segmentation.
• Address limitations in remote modeling capabilities and computational complexity of current fusion techniques.
• Leverage dual-branch encoding with Mamba for enhanced global feature extraction.

Main Methods:

• Propose a dual-branch encoding structure: a CNN main encoder for High-Resolution Remote Sensing Images (HRRSIs) and a Mamba auxiliary encoder for Digital Surface Models (DSMs).
• Design a Feature Fusion Block (FFB) for synergistic feature enhancement and integration from dual branches.
• Utilize Mamba's capabilities for efficient global feature capture and CNNs for local feature extraction.

Main Results:

• MFMamba demonstrates superior performance on Vaihingen and Potsdam datasets.
• Achieved higher Overall Accuracy (OA), mean F1 score (mF1), and mean Intersection over Union (mIoU) compared to state-of-the-art methods.
• Maintained low computational complexity while achieving high segmentation accuracy.

Conclusions:

• MFMamba effectively integrates multi-modal remote sensing data for improved semantic segmentation.
• The Mamba-based approach offers a promising direction for efficient and accurate remote sensing image analysis.
• MFMamba sets a new benchmark for semantic segmentation in remote sensing applications.