SCSV: Spatial-Temporal Consistent Dynamic 3D Scene Generation From Sparse Views.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Generating dynamic scenes from limited images presents challenges.
• Existing methods struggle with sparse data and maintaining spatial-temporal consistency in multi-view videos.

Purpose of the Study:

• To propose SCSV, a method for spatial-temporally consistent dynamic scene generation from sparse views.
• To address limitations in handling sparse images and ensuring consistency in multi-view video rendering.

Main Methods:

• SCSV employs a two-stage approach: scene reconstruction and scene expansion, decoupling background and foreground.
• Scene reconstruction involves image interpolation and uncertainty-aware Gaussian optimization.
• Scene expansion refines background views with diffusion models and generates coherent foreground motion, integrating them with gravity-aware alignment.

Main Results:

• SCSV demonstrates superior performance on NeuMan, Bonn, and EMDB datasets compared to state-of-the-art methods.
• The method effectively handles sparse images and maintains spatial-temporal consistency.
• Achieved physically plausible dynamic scene generation.

Conclusions:

• SCSV offers a robust solution for dynamic scene generation from sparse inputs.
• The proposed uncertainty-aware training and geometry-aware diffusion enhance scene quality and consistency.
• This work advances the field of dynamic scene synthesis.