The Adversarial Robust and Generalizable Stereo Matching for Infrared Binocular Based on Deep Learning

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Deep learning excels in stereo matching but struggles with generalizability and robustness, especially with infrared (IR) textures and occlusions.
• Existing methods often require extensive IR datasets and lack adaptability to different IR cameras.

Purpose of the Study:

• To develop a novel deep-learning-based depth optimization method for stereo matching.
• To enhance robustness and generalizability across various imaging conditions and datasets.
• To investigate the utility of infrared textures in deep learning stereo matching.

Main Methods:

• Utilizing a multi-scale census transform for computing the matching cost volume.
• Employing a stacked sand leak subnetwork for the stereo matching task.
• Adapting the method for both infrared and standard binocular images without large datasets.

Main Results:

• Achieved substantial improvements in adversarial robustness while maintaining accuracy.
• Reduced endpoint error (EPE) by nearly half compared to state-of-the-art methods on autonomous driving datasets.
• Demonstrated superior generalization from simulated to real-world datasets without fine-tuning.

Conclusions:

• The proposed method offers a robust and adaptable solution for stereo matching, effective across diverse conditions and imaging types.
• Infrared textures remain valuable for stereo matching within deep learning frameworks, even in challenging lighting.
• The approach significantly advances stereo matching capabilities for applications like autonomous driving.