MaskScene: Hierarchical Conditional Masked Models for Real-Time 3D Indoor Scene Synthesis.

Area of Science:

• Computer Vision
• Artificial Intelligence
• 3D Graphics

Background:

• Diffusion and autoregressive models show promise for scene synthesis.
• Existing methods face challenges in balancing real-time performance, visual fidelity, and editability.

Purpose of the Study:

• To propose MaskScene, a novel hierarchical conditional masked model for efficient and editable 3D indoor scene synthesis.
• To overcome limitations of current models in speed, quality, and flexibility.

Main Methods:

• Introduced a hierarchical scene representation encoding relationships, semantics, and tokenization.
• Designed a hierarchical conditional masked modeling architecture for parallel and iterative decoding.
• Utilized masking of local objects and hierarchical structure for inferring missing scene regions.

Main Results:

• Achieved 80x faster generation speed compared to state-of-the-art methods.
• Improved scene quality by 10%.
• Enabled zero-shot editing capabilities like scene completion and rearrangement without fine-tuning.

Conclusions:

• MaskScene offers a significant advancement in real-time 3D indoor scene synthesis and editing.
• The model's hierarchical approach enhances efficiency, quality, and editability.
• Future work includes public release of the project and dataset.