OD-MVSNet: Omni-dimensional dynamic multi-view stereo network.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Multi-view stereo (MVS) is crucial for 3D reconstruction, but current methods struggle with precision due to feature extraction limitations and cost-volume correlation issues.
• Accurate inference of depth maps and fine-grained scene geometry remains a significant challenge in MVS.

Purpose of the Study:

• To propose a novel cascade deep residual inference network, OD-MVSNet, to enhance the efficiency and accuracy of multi-view stereo depth estimation.
• To improve the precision of 3D reconstruction by addressing feature extraction and cost-volume correlation problems.

Main Methods:

• Developed a cost-volume pyramid approach, building from coarse to fine for a lightweight and compact network.
• Introduced the omni-dimensional dynamic atrous spatial pyramid pooling (OSPP) for multiscale feature extraction and dense feature map generation.
• Proposed a normalization-based 3D attention module to aggregate crucial information within the cost volume, mitigating feature mismatch.

Main Results:

• The OD-MVSNet model demonstrated superior performance on the DTU benchmark dataset compared to baseline models.
• Achieved approximately 1.4% reduction in accuracy loss, 0.9% in completeness loss, and 1.2% in overall loss.

Conclusions:

• The proposed OD-MVSNet effectively enhances multi-view stereo depth estimation for high-accuracy 3D reconstruction.
• The OSPP and 3D attention modules contribute significantly to improved feature extraction and cost volume regularization, enabling dense point cloud generation with reduced memory.