OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Autonomous driving systems require understanding and predicting 3D scene evolution.
• Current world models often use autoregressive methods, leading to error accumulation and poor long-term consistency.
• Generative frameworks are needed for robust modeling of dynamic 3D environments.

Purpose of the Study:

• To develop a diffusion-based 4D occupancy generation model for simulating 3D scene evolution in autonomous driving.
• To overcome limitations of autoregressive models in long-term consistency and global spatiotemporal reasoning.
• To create a world simulator for autonomous driving decision-making.

Main Methods:

• Introduced a 4D scene tokenizer for compact spatiotemporal representations.
• Developed a diffusion transformer model trained on these representations.
• Generated 4D occupancy conditioned on trajectory prompts for simulation.

Main Results:

• OccSora successfully generated 16-second videos with authentic 3D layout and strong temporal consistency.
• The model demonstrated high-quality reconstruction of long occupancy sequences.
• Achieved trajectory-aware 4D generation for realistic scene simulation.

Conclusions:

• OccSora offers a novel diffusion-based approach for simulating 3D world evolution.
• The model significantly improves long-term consistency in generated scene dynamics.
• OccSora shows potential as a world simulator for enhancing autonomous driving decision-making.