ActiveZero++: Mixed Domain Learning Stereo and Confidence-Based Depth Completion With Zero Annotation.

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Learning-based stereo methods typically demand extensive real-world depth data, which is challenging to acquire accurately.
• Simulation environments offer readily available ground truth depth, but bridging the Sim2Real gap remains a significant hurdle.

Purpose of the Study:

• To introduce ActiveZero++, a mixed-domain learning framework for active stereovision that bypasses the need for real-world depth annotations.
• To enhance depth prediction accuracy and robustness in challenging regions using novel self-supervised techniques.

Main Methods:

• Utilizes a combination of supervised and self-supervised losses in the simulation domain on a shape primitives dataset.
• Employs self-supervised loss on out-of-distribution real-world data, introducing a temporal infrared (IR) reprojection loss for improved robustness.
• Incorporates a confidence-based depth completion module that leverages stereo network confidence and depth-normal consistency for error correction.

Main Results:

• Achieves state-of-the-art performance on real-world data, surpassing commercial depth sensors in qualitative and quantitative evaluations.
• Significantly reduces the Sim2Real domain gap for depth maps, benefiting downstream tasks like 6D pose estimation.
• Demonstrates improved accuracy and robustness in depth prediction, particularly in difficult-to-perceive areas.

Conclusions:

• ActiveZero++ presents an effective solution for active stereovision by leveraging mixed-domain learning and novel self-supervised losses.
• The framework successfully addresses the limitations of real-world depth data acquisition and the Sim2Real domain gap.
• This approach offers a promising direction for advancing depth estimation in real-world applications without requiring extensive manual annotation.