CMANet: Cross-Modality Attention Network for Indoor-Scene Semantic Segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Indoor-scene semantic segmentation is crucial for applications like navigation and mapping.
• Integrating RGB and HHA data presents challenges due to differing physical properties and complex indoor structures.
• Existing methods struggle with effective multi-modal feature fusion for indoor environments.

Purpose of the Study:

• To propose a novel network, Cross-Modality Attention Network (CMANet), for enhanced indoor-scene semantic segmentation.
• To improve the extraction and integration of features from both RGB and HHA image modalities.
• To achieve superior performance in semantic segmentation tasks by effectively fusing multi-modal data.

Main Methods:

• Developed an encoder-decoder architecture with parallel branches for RGB and HHA feature extraction.
• Introduced a Cross-Modality Refine Gate (CMRG) utilizing self-attention for feature fusion.
• Employed a multi-stage decoder with residual blocks, bi-directional propagation, and pyramid supervision.

Main Results:

• The proposed CMANet effectively extracts and fuses features from RGB and HHA images.
• Experimental results on NYUDv2 and SUN RGB-D datasets show superior performance compared to existing methods.
• The CMRG module is identified as a key component for successful cross-modality fusion.

Conclusions:

• CMANet demonstrates significant effectiveness and efficiency for indoor-scene semantic segmentation.
• The proposed method advances the state-of-the-art in multi-modal indoor scene understanding.
• The approach offers a robust solution for tasks requiring accurate indoor spatial awareness.