Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method

Area of Science:

• Computer Vision
• Autonomous Driving Systems
• Machine Learning for Robotics

Background:

• Occupancy-centric methods excel in autonomous driving scene generation but require extensive annotated data, which is scarce.
• Existing datasets limit the scale and diversity needed for advanced generative modeling and downstream task evaluation.

Purpose of the Study:

• To address the scarcity of semantic occupancy data for autonomous driving.
• To develop a unified framework for high-fidelity generation of semantic occupancy, multi-view videos, and LiDAR point clouds.
• To enhance the performance of downstream autonomous driving applications like perception and planning.

Main Methods:

• Curated Nuplan-Occ, the largest semantic occupancy dataset from the Nuplan benchmark.
• Developed a spatio-temporal disentangled architecture for 4D dynamic occupancy forecasting.
• Proposed Gaussian splatting-based rendering for video generation and sensor-aware embeddings for LiDAR simulation.

Main Results:

• Achieved superior generation fidelity and scalability compared to existing methods.
• Demonstrated practical value in downstream autonomous driving tasks.
• Successfully synthesized high-quality semantic occupancy, multi-view videos, and LiDAR point clouds.

Conclusions:

• The Nuplan-Occ dataset and the proposed unified framework significantly advance generative modeling for autonomous driving.
• The method provides a scalable solution for generating diverse and realistic driving scenes, crucial for robust system evaluation.