NR-MVSNet: Learning Multi-View Stereo Based on Normal Consistency and Depth Refinement.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Learning-based Multi-view Stereo (MVS) methods show promise but suffer from accumulative errors and inaccurate depth hypotheses.
• Existing methods struggle with texture-less and repetitive-texture regions, limiting 3D model quality.

Purpose of the Study:

• To propose NR-MVSNet, a novel Multi-view Stereo network designed to overcome limitations of current learning-based approaches.
• To enhance the accuracy and robustness of 3D point cloud reconstruction from multiple views.

Main Methods:

• Introduced a Depth Hypotheses based on Normal Consistency (DHNC) module for generating more effective depth hypotheses by leveraging neighboring pixel normals.
• Developed a Depth Refinement with Reliable Attention (DRRA) module to update initial depth maps, integrating attentional and cost volume features to mitigate accumulative errors.

Main Results:

• The DHNC module produces smoother and more accurate depth predictions, particularly in challenging regions.
• The DRRA module improves coarse-stage depth estimation accuracy, effectively addressing accumulative error issues.
• NR-MVSNet demonstrated superior efficiency and robustness across multiple benchmark datasets (DTU, BlendedMVS, Tanks & Temples, ETH3D).

Conclusions:

• NR-MVSNet significantly advances Multi-view Stereo reconstruction by introducing novel modules for improved depth hypothesis generation and refinement.
• The proposed method offers a more accurate and robust solution for 3D point cloud reconstruction compared to state-of-the-art techniques.