End-to-End Learning for Omnidirectional Stereo Matching With Uncertainty Prior

Area of Science:

• Computer Vision
• Deep Learning
• 3D Reconstruction

Background:

• Omnidirectional depth estimation is crucial for robotics and augmented reality.
• Existing multi-view stereo methods often struggle with wide-baseline and ultra-wide field-of-view imagery.

Purpose of the Study:

• To develop a novel end-to-end deep neural network for accurate omnidirectional depth estimation.
• To introduce uncertainty prior guidance for improved depth map accuracy.
• To present large-scale synthetic datasets for training and evaluating omnidirectional multi-view stereo algorithms.

Main Methods:

• A deep neural network model processing ultra-wide field-of-view images from an omnidirectional rig.
• Feature maps are warped onto concentric spheres using calibrated camera parameters.
• A 3D encoder-decoder block generates depth estimates, with regularization for uncertain regions.
• Uncertainty prior guidance is applied through depth map filtering and regularization.

Main Results:

• The proposed method achieves excellent omnidirectional depth estimation in both synthetic and real-world environments.
• The model outperforms prior art and conventional state-of-the-art stereo algorithms adapted for omnidirectional data.
• Large-scale synthetic datasets with 13K ground-truth depth maps and 53K fisheye images were created.

Conclusions:

• The novel deep neural network effectively addresses omnidirectional depth estimation challenges.
• Uncertainty guidance significantly enhances depth map accuracy.
• The introduced datasets facilitate further research in omnidirectional multi-view stereo.