Street Gaussians: Modeling Dynamic Urban Scenes With Gaussian Primitives

Area of Science:

• Computer Vision
• Robotics
• 3D Scene Representation

Background:

• Modeling dynamic urban street scenes is crucial for autonomous driving.
• Existing methods like Neural Radiance Fields (NeRF) offer photo-realistic synthesis but suffer from slow training and rendering.
• Limitations in speed hinder real-time application and scalability for autonomous driving systems.

Purpose of the Study:

• To address the limitations of slow training and rendering speeds in dynamic urban scene modeling.
• To introduce a novel explicit scene representation for efficient and high-quality view synthesis.
• To enable faster scene editing and rendering for autonomous driving applications.

Main Methods:

• Introduced Street Gaussians, an explicit scene representation using point clouds with semantic logits and Gaussian primitives.
• Modeled foreground object dynamics through optimizable tracked poses and a 4D spherical harmonics model for appearance.
• Enabled easy composition of objects and background for scene editing and rendering.

Main Results:

• Achieved rendering speeds of 135 FPS at 1066*1600 resolution.
• Completed training in under half an hour.
• Demonstrated superior performance over state-of-the-art methods on KITTI and Waymo Open datasets.

Conclusions:

• Street Gaussians provide an efficient and effective solution for modeling dynamic urban scenes for autonomous driving.
• The explicit representation facilitates faster training, rendering, and scene editing.
• The method shows significant improvements in performance and speed compared to previous approaches.