Stereo Confidence Estimation via Locally Adaptive Fusion and Knowledge Distillation.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Stereo confidence estimation is crucial for assessing disparity accuracy in stereo matching.
• Previous methods often rely on limited input modalities, hindering performance.
• Developing robust methods using comprehensive input is essential for reliable 3D reconstruction.

Purpose of the Study:

• To propose a novel method for stereo confidence estimation using tri-modal input.
• To introduce Locally Adaptive Fusion Networks (LAF-Net) for effective feature fusion.
• To develop a knowledge distillation framework for compact and efficient confidence estimation networks.

Main Methods:

• Utilizing deep networks to process tri-modal input: matching cost, disparity, and color image.
• Implementing LAF-Net with locally-varying attention and scale maps for feature fusion.
• Employing a knowledge distillation framework with a locally-varying temperature softmax module.

Main Results:

• LAF-Net effectively fuses tri-modal confidence features, leading to superior performance.
• Knowledge distillation allows student networks (disparity input only) to achieve comparable results.
• The proposed methods outperform state-of-the-art stereo confidence estimation techniques on benchmarks.

Conclusions:

• LAF-Net offers a powerful approach to stereo confidence estimation by leveraging multi-modal data.
• Knowledge distillation provides an effective way to create efficient yet accurate confidence estimation models.
• The framework is extendable to multiview scenarios, demonstrating broad applicability.