Visibility-Aware Multi-View Stereo by Surface Normal Weighting for Occlusion Robustness.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Learning-based Multi-View Stereo (MVS) methods struggle with accuracy in large occlusion scenarios due to noise from mismatched features.
• Significant inter-camera distances or complex object shapes exacerbate these occlusion challenges in MVS.

Purpose of the Study:

• To develop a visibility-aware MVS method that enhances accuracy and robustness in occluded environments.
• To introduce novel techniques for cost volume construction and regularization using explicit 3D geometry.

Main Methods:

• Proposed SnowMVSNet, incorporating inter-view visibility to suppress mismatched features during cost volume construction.
• Introduced geometry-guided cost volume regularization using surface normal priors to enhance true depth.
• Developed intra-view visibility for distinguishing geometrically visible pixels and enabling visibility-weighted training and depth estimation.

Main Results:

• SnowMVSNet demonstrated substantial performance improvements, particularly in occlusion robustness, relative to computational complexity.
• The method achieved accurate 3D point cloud reconstruction by focusing on visible regions.
• Outperformed state-of-the-art methods in both low- and high-occlusion scenarios on the MVHuman dataset.

Conclusions:

• SnowMVSNet effectively addresses MVS limitations in occluded scenarios through simple yet powerful visibility computations.
• The proposed visibility-aware approach offers a robust solution for accurate 3D reconstruction in challenging environments.
• This work provides a significant advancement in occlusion robustness for learning-based MVS techniques.