Unsupervised 3D Reconstruction with Multi-Measure and High-Resolution Loss

Area of Science:

• Computer Vision
• Deep Learning
• 3D Reconstruction

Background:

• Deep learning-based multi-view 3D reconstruction is advancing rapidly.
• Unsupervised learning is a key area due to the absence of ground truth labels.
• Current unsupervised methods using 3D Convolutional Neural Networks (3DCNN) are computationally intensive and require substantial memory.

Purpose of the Study:

• To propose an end-to-end unsupervised multi-view 3D reconstruction network framework.
• To significantly reduce memory requirements and computing time compared to existing methods.
• To improve the accuracy and quality of high-resolution 3D reconstructions.

Main Methods:

• Developed Unsup_patchmatchnet, an unsupervised framework leveraging PatchMatch.
• Introduced a novel feature point consistency loss function.
• Incorporated self-supervised signals including photometric and semantic consistency losses.
• Implemented a high-resolution loss method to enhance reconstruction detail.

Main Results:

• Achieved an 80% reduction in memory usage and over 50% reduction in running time compared to 3DCNN methods.
• Attained a low overall error of 0.501 mm for reconstructed 3D point clouds.
• Demonstrated superior performance against most current unsupervised multi-view 3D reconstruction networks.
• Verified good generalization capabilities across different datasets.

Conclusions:

• Unsup_patchmatchnet offers a highly efficient and accurate solution for unsupervised multi-view 3D reconstruction.
• The proposed loss functions and high-resolution method effectively improve reconstruction quality.
• The network shows strong performance and generalization, outperforming existing state-of-the-art methods.