DecoRec: Decomposed 3D Scene Reconstruction From Single-View Images via Object-Level Diffusion

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computer Graphics

Background:

• Single-view 2D image to 3D scene reconstruction is challenging due to limitations in current methods.
• Existing approaches often result in inaccurate geometry and appearance.
• Lack of large-scale scene-level datasets hinders direct 3D scene generation.

Purpose of the Study:

• Introduce DecoRec, a novel system for high-quality single-view 3D scene reconstruction.
• Overcome limitations of existing methods in capturing accurate geometry and appearance.
• Provide a robust solution for generating 3D scene meshes from single 2D images.

Main Methods:

• Leverage diffusion-based single-view object reconstruction for individual object generation.
• Employ a refinement pipeline to merge reconstructed objects.
• Utilize differentiable rendering and diffusion-guided refinement for enhanced appearance and geometry.

Main Results:

• DecoRec achieves high-quality single-view scene reconstruction.
• Demonstrates significant improvements in both geometric accuracy and novel view synthesis.
• Successfully generates decomposed 3D scene meshes from single 2D images.

Conclusions:

• DecoRec offers a significant advancement in single-view 3D scene reconstruction.
• The system shows promise for applications such as interior design.
• Future work may involve further refinement and dataset expansion.