Learning-Based Multi-View Stereo: A Survey

Area of Science:

• Computer Vision
• 3D Geometry Processing
• Machine Learning

Background:

• 3D reconstruction is crucial for applications like Augmented/Virtual Reality (AR/VR), autonomous driving, and robotics.
• Multi-View Stereo (MVS) algorithms leverage multiple viewpoints for comprehensive 3D scene representation and are pivotal for image-based reconstruction.
• Deep learning has significantly advanced MVS methods, outperforming traditional approaches.

Purpose of the Study:

• To provide a comprehensive review of learning-based Multi-View Stereo (MVS) methods.
• To categorize existing learning-based MVS approaches.
• To identify and discuss future research directions in the field of 3D reconstruction.

Main Methods:

• Categorization of learning-based MVS methods into depth map-based, voxel-based, NeRF-based, 3D Gaussian Splatting-based, and large feed-forward methods.
• In-depth focus on depth map-based MVS methods, highlighting their advantages in conciseness, flexibility, and scalability.
• Literature review and performance summarization on popular benchmarks.

Main Results:

• Learning-based MVS methods demonstrate impressive performance compared to traditional techniques.
• Depth map-based methods are identified as a primary family within MVS due to their practical advantages.
• A categorization framework is established for understanding the landscape of modern MVS research.

Conclusions:

• Learning-based MVS methods represent the state-of-the-art in image-based 3D reconstruction.
• Depth map-based approaches are particularly promising for future development in MVS.
• Further research is needed to explore novel architectures and improve reconstruction accuracy and efficiency.