BSI-MVS: multi-view stereo network with bidirectional semantic information.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Multi-view stereo (MVS) reconstructs 3D scenes from multiple images.
• Current state-of-the-art MVS networks often rely on Vision Transformers, leading to high computational costs.
• Improving efficiency and accuracy in MVS is crucial for practical applications.

Purpose of the Study:

• To develop a novel MVS network that reduces computational complexity.
• To enhance the accuracy of depth map generation in MVS.
• To introduce a method that effectively captures semantic information for 3D reconstruction.

Main Methods:

• Proposed a Bidirectional Semantic Information (BSI-MVS) network.
• Designed a Multi-Level Spatial Pyramid module for multi-scale feature extraction.
• Implemented a 2D Bidirectional-LSTM module to capture semantic context.
• Utilized cost volumes built on multi-level features for depth map optimization.

Main Results:

• The BSI-MVS network demonstrated superior performance compared to existing methods.
• Achieved significant improvements over TransMVSNet (17.84%), CasMVSNet (36.42%), CVP-MVSNet (14.96%), and AACVP-MVSNet (4.86%).
• Showcased noticeable enhancements in both objective metrics and visual quality of reconstructed depth maps.

Conclusions:

• The BSI-MVS network effectively reduces computational complexity in MVS.
• The proposed method significantly improves depth map accuracy.
• BSI-MVS offers a promising approach for efficient and accurate 3D reconstruction.