Polarization-driven semantic segmentation via efficient attention-bridged fusion.

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Semantic segmentation (SS) is crucial for outdoor scene perception in safety-critical systems.
• Traditional RGB-based SS struggles in complex environments due to limited information dimensions.
• The necessity for multimodal fusion is evident in scenarios like unexpected obstacle detection.

Purpose of the Study:

• To develop an efficient network for fusing complementary information from multiple optical sensors.
• To enhance semantic segmentation reliability in challenging outdoor conditions.
• To investigate the utility of polarization sensing for improved scene understanding.

Main Methods:

• Proposed EAFNet (Efficient Attention-bridged Fusion Network) for multimodal sensor fusion.
• Incorporated polarization sensing to capture supplementary material information.
• Created the first RGB-P dataset with 394 annotated pixel-aligned RGB-polarization images using a single-shot sensor.

Main Results:

• Demonstrated the effectiveness of EAFNet in fusing polarization and RGB information.
• Achieved improved semantic segmentation performance by leveraging multimodal data.
• Showcased the network's flexibility for adaptation to other sensor combinations.

Conclusions:

• Multimodal fusion, particularly incorporating polarization data, significantly enhances outdoor scene perception.
• EAFNet provides an efficient and flexible solution for robust semantic segmentation.
• The developed RGB-P dataset facilitates further research in multimodal perception for autonomous systems.