CSANet: Context-Spatial Awareness Network for RGB-T Urban Scene Understanding

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Semantic segmentation is vital for urban scene understanding in autonomous driving.
• Existing methods struggle with low-light and adverse weather, limiting real-world application.
• Integrating RGB and thermal infrared (TIR) data offers a promising solution.

Purpose of the Study:

• To develop a novel framework, CSANet, for robust RGB-T semantic segmentation.
• To enhance feature extraction and fusion for improved accuracy in challenging conditions.
• To advance the capabilities of autonomous driving systems.

Main Methods:

• CSANet utilizes an efficient encoder for local and global feature extraction.
• A hierarchical fusion strategy selectively integrates visual and semantic information.
• Key modules include Channel-Spatial Cross-Fusion (CSCFM), Multi-Head Fusion (MHFM), and Spatial Coordinate Attention (SCAM).

Main Results:

• CSANet demonstrates state-of-the-art performance on benchmark datasets (MFNet, PST900).
• The framework effectively fuses RGB and TIR modalities for superior semantic segmentation.
• Significant improvements in object localization accuracy were observed.

Conclusions:

• CSANet offers a robust solution for RGB-T semantic segmentation, especially in adverse conditions.
• The proposed fusion strategies and attention mechanisms enhance understanding of complex urban environments.
• This work contributes to safer and more reliable autonomous driving systems.