Dust to Tower: Prior-Driven Coarse-to-Fine Photo-Realistic Scene Reconstruction From Sparse Uncalibrated Images.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Photogrammetry

Background:

• Photo-realistic 3D scene reconstruction from sparse, uncalibrated images is a challenging problem.
• Existing methods either require accurate camera parameters or densely captured images, limiting their practical application.

Purpose of the Study:

• To propose a novel coarse-to-fine framework, Dust to Tower (D2T), to address the coupled challenges of sparse-view and uncalibrated 3D reconstruction.
• To enable reliable supervision at novel viewpoints without relying on computationally expensive view synthesis techniques.

Main Methods:

• The framework employs a Coarse Construction Module (CCM) using a fast Multi-View Stereo model for initial 3D Gaussian Splatting (3DGS) and camera pose recovery.
• Confidence-Aware Depth Alignment (CADA) refines the 3D model by aligning monocular inverse-depth priors to reliable regions using DUSt3R confidence.
• Warped Image-Guided Inpainting (WIGI) generates multi-view-consistent pseudo supervision from accurately warped views.

Main Results:

• D2T demonstrates superior novel view synthesis quality compared to ten representative baselines.
• The method achieves higher pose accuracy than existing approaches.
• Experiments on three benchmark datasets validate the effectiveness and efficiency of the proposed framework.

Conclusions:

• The Dust to Tower (D2T) framework offers an effective solution for photo-realistic 3D scene reconstruction from sparse, uncalibrated images.
• D2T achieves state-of-the-art performance in novel view synthesis and pose accuracy while maintaining high efficiency.
• The proposed methods, CCM, CADA, and WIGI, contribute to narrowing the solution space and introducing reliable supervision for improved reconstruction.