DrivingGaussian++: Toward Realistic Reconstruction and Editable Simulation for Surrounding Dynamic Driving Scenes

Area of Science:

• Computer Vision
• Robotics
• 3D Scene Reconstruction

Background:

• Autonomous driving systems require accurate perception of dynamic environments.
• Existing methods struggle with realistic reconstruction and controllable editing of complex dynamic scenes.

Purpose of the Study:

• To develop an efficient and effective framework for realistic reconstruction and controllable editing of dynamic autonomous driving scenes.
• To improve the accuracy, consistency, and realism of 3D scene reconstruction and synthesis.

Main Methods:

• Utilizes incremental 3D Gaussians for static background and a composite dynamic Gaussian graph for moving objects.
• Integrates a LiDAR prior for detailed and consistent scene reconstruction.
• Employs multi-view images and depth priors for training-free controllable editing.
• Incorporates large language models (LLMs) for automatic generation and enhancement of dynamic object motion trajectories.

Main Results:

• Achieves state-of-the-art performance in dynamic scene reconstruction and photorealistic surround-view synthesis.
• Enables training-free controllable editing, including texture modification, weather simulation, and object manipulation.
• Demonstrates consistent and realistic editing results, enhancing scene diversity and realism.

Conclusions:

• DrivingGaussian++ provides a robust solution for realistic 3D reconstruction and editing of dynamic driving scenes.
• The integration of LLMs significantly enhances the realism and controllability of dynamic scene generation.
• This framework advances the capabilities for creating diverse and dynamic multi-view driving scenarios.