Neural Disparity Refinement.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Disparity map generation is crucial for 3D reconstruction and depth perception.
• Existing methods often struggle with high-resolution outputs or generalization across different data sources.
• Unbalanced stereo setups, common in mobile devices, pose additional challenges.

Purpose of the Study:

• To develop a versatile framework for enhancing disparity maps from various sources.
• To achieve high-quality, high-resolution disparity map refinement.
• To enable robust performance across diverse stereo imaging conditions, including unbalanced setups.

Main Methods:

• A neural disparity refinement network employing a continuous feature sampling strategy.
• Integration of traditional hand-crafted algorithms with deep learning techniques.
• Framework designed to process disparity maps from classical stereo algorithms, modern neural networks, and structure-from-motion pipelines.

Main Results:

• The proposed framework successfully generates high-quality, high-resolution disparity maps.
• Demonstrated strong zero-shot generalization capabilities from synthetic to real images when combined with classical stereo algorithms.
• Effectively handles unbalanced stereo setups, a common issue in mobile phone cameras.

Conclusions:

• The neural disparity refinement network offers a flexible and powerful solution for enhancing stereo image disparity.
• The continuous feature sampling approach allows for arbitrary output resolutions and improved generalization.
• The framework's adaptability makes it suitable for a wide range of applications, including mobile depth sensing.