SceneDreamer: Unbounded 3D Scene Generation From 2D Image Collections.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Generating large-scale 3D scenes from scratch is challenging.
• Existing methods often require 3D annotations, limiting their applicability.
• Synthesizing complex, unbounded 3D environments from limited data remains an open problem.

Purpose of the Study:

• To introduce SceneDreamer, an unconditional generative model for creating unbounded 3D scenes.
• To develop a framework that learns 3D scene synthesis solely from 2D image collections.
• To achieve efficient and expressive 3D scene representation and generation.

Main Methods:

• Utilizes an efficient bird's-eye-view (BEV) representation with height and semantic fields.
• Employs a generative neural hash grid for scene parameterization based on 3D position and semantics.
• Leverages a neural volumetric renderer trained with adversarial methods on 2D images.

Main Results:

• SceneDreamer synthesizes large-scale, unbounded 3D landscapes from random noise.
• The model demonstrates effective disentanglement of geometry and semantics.
• Achieves state-of-the-art performance in generating vivid and diverse 3D worlds.

Conclusions:

• SceneDreamer offers a novel solution for unconditional 3D scene generation.
• The framework successfully learns 3D scene synthesis from 2D data alone.
• The approach shows superiority over existing methods in creating realistic and varied 3D environments.