SceneLinker: Compositional 3D Scene Generation via Semantic Scene Graph from RGB Sequences.

Area of Science:

• Computer Vision
• 3D Scene Reconstruction
• Mixed Reality

Background:

• Generating 3D scenes that reflect real-world layouts is crucial for adaptive Mixed Reality (MR) experiences.
• Previous methods often fail to capture contextual object relationships or prioritize shape synthesis over arrangement accuracy.

Purpose of the Study:

• To introduce SceneLinker, a novel framework for generating compositional 3D scenes from RGB sequences using semantic scene graphs.
• To improve the alignment of 3D scenes with object arrangements and real-world layouts.

Main Methods:

• Developed a graph network with cross-check feature attention for scene graph prediction.
• Constructed a graph-variational autoencoder (graph-VAE) with a joint shape and layout block for 3D scene generation.

Main Results:

• SceneLinker outperforms state-of-the-art methods in quantitative and qualitative evaluations on benchmark datasets (3RScan/3DSSG, SG-FRONT).
• The framework demonstrates effectiveness in complex indoor environments and under challenging scene graph constraints.

Conclusions:

• SceneLinker enables the generation of consistent 3D spaces from physical environments via scene graphs.
• This facilitates the creation of spatial MR content tailored to user environments.