ReconX: Reconstruct Any Scene From Sparse Views With Video Diffusion Model.

Area of Science:

• Computer Vision
• 3D Graphics
• Artificial Intelligence

Background:

• 2D image to 3D model reconstruction shows great success in dense-view scenarios.
• Sparse-view 3D reconstruction remains an ill-posed problem, leading to artifacts and distortions.

Purpose of the Study:

• Propose ReconX, a novel paradigm for 3D scene reconstruction from sparse views.
• Address the challenge of 3D view consistency in generative models for reconstruction.

Main Methods:

• Reframe sparse-view reconstruction as a temporal generation task using pre-trained video diffusion models.
• Encode a global point cloud as a 3D structure condition to guide video frame synthesis.
• Employ confidence-aware 3D Gaussian Splatting for final 3D scene recovery from generated video.

Main Results:

• ReconX synthesizes detail-preserved video frames with high 3D consistency from limited input views.
• The method effectively overcomes artifacts and distortions common in sparse-view reconstruction.
• Demonstrates superior quality and generalizability compared to state-of-the-art methods on real-world datasets.

Conclusions:

• ReconX offers a powerful new approach for 3D scene reconstruction from sparse views.
• Leveraging generative priors from video diffusion models enhances reconstruction accuracy and consistency.
• The proposed method advances the field of 3D reconstruction, particularly for challenging sparse-view scenarios.