Dynamics-Aware Gaussian Splatting Streaming Toward Fast On-the-Fly 4D Reconstruction

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computer Graphics

Background:

• 3D Gaussian splatting (3DGS) shows promise for 4D dynamic spatial reconstruction.
• Current 3DGS streaming methods lack temporal continuity and uniform primitive handling.
• Limited research exists on online reconstruction for on-the-fly training and per-timestep streaming.

Purpose of the Study:

• To develop an iterative streamable 4D dynamic spatial reconstruction pipeline.
• To address limitations in current 3DGS streaming methods regarding temporal continuity and feature differentiation.
• To enable efficient on-the-fly training and per-timestep streaming for dynamic scenes.

Main Methods:

• A novel pipeline with three stages: selective inheritance, dynamics-aware shift, and error-guided densification.
• Selective inheritance preserves temporal continuity by retaining priors from previous timesteps.
• Dynamics-aware shift distinguishes and optimizes dynamic and static primitives separately.
• Error-guided densification efficiently identifies Gaussians needing updates for new objects.

Main Results:

• Achieved state-of-the-art performance in online 4D reconstruction.
• Demonstrated compact storage requirements.
• Showcased the fastest on-the-fly training speeds.
• Delivered superior scene representation quality.

Conclusions:

• The proposed pipeline effectively addresses limitations in existing 3DGS streaming methods.
• The method enables efficient and high-quality online 4D dynamic spatial reconstruction.
• Future work can explore further optimizations for real-time applications.