Simple Scalable Multimodal Semantic Segmentation Model.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Visual perception is vital for autonomous driving.
• Traditional methods use single-modal RGB images for semantic segmentation.
• Multimodal approaches offer complementary data for enhanced robustness.

Purpose of the Study:

• To develop a multimodal visual perception model for autonomous driving.
• To address the accuracy decline observed when adding modalities to traditional models.
• To create a scalable model that maintains or improves accuracy with any modality.

Main Methods:

• Proposed a multimodal model with an RGB main branch and shared backbone for other modalities.
• Introduced the Modals Score Module (MSM) to evaluate feature importance.
• Integrated the Features Complementary Module (FCM) for enhanced feature fusion.
• Applied residual thinking to improve feature extraction across all branches.

Main Results:

• Adding modalities to traditional models can decrease semantic segmentation accuracy.
• The proposed multimodal model maintains or improves accuracy with any additional modality.
• The model surpasses current state-of-the-art multimodal semantic segmentation approaches.

Conclusions:

• The proposed multimodal model is effective and scalable for autonomous driving visual perception.
• MSM, FCM, and residual thinking significantly contribute to the model's performance.
• This approach offers a robust solution for enhancing semantic segmentation in autonomous systems.