Planner3D: LLM-enhanced Graph Prior Meets 3D Indoor Scene Explicit Regularization

Area of Science:

• Computer Vision
• Artificial Intelligence
• 3D Graphics

Background:

• Compositional 3D scene synthesis is crucial for robotics, film, and gaming, but current methods struggle with shape diversity and scene-level fidelity.
• Existing generative models improve shape fidelity but often neglect joint layout-shape generation, leading to issues like object collisions.

Purpose of the Study:

• To develop a method for generating realistic and reasonable 3D indoor scenes directly from scene graphs.
• To address the under-explored area of scene-level fidelity in 3D scene synthesis.

Main Methods:

• Utilized a large language model to enrich scene graph priors by aggregating global, node-wise, and edge-wise features.
• Employed a unified graph encoder to extract features for guiding joint layout-shape generation.
• Introduced additional regularization to explicitly constrain the generated 3D layouts.

Main Results:

• Achieved improved 3D scene synthesis performance, particularly in scene-level fidelity, when benchmarked on the SG-FRONT dataset.
• Demonstrated the effectiveness of integrating large language models and generative models for complex scene generation.

Conclusions:

• The proposed method successfully generates realistic 3D indoor scenes by effectively combining scene graph information with generative capabilities.
• The approach offers a significant advancement in synthesizing complex, collision-free 3D environments with high scene-level fidelity.