Scene as Occupancy and Reconstruction: A Comprehensive Dataset for Unstructured Scene Understanding

As autonomous driving technology steps into the phase of large-scale commercialization, safety and comfort have become key indicators for measuring its performance. Currently, some studies have begun to focus on improving the safety and comfort of urban driving by paying attention to irregular surface regions. However, datasets and studies for unstructured scenes, which are characterized by numerous irregular obstacles and road surface undulations, remain exceedingly rare. To expand the scope of autonomous driving applications, a perception dataset, which focuses on irregular obstacles and road surface vibrations in unstructured scenes, has been built. It takes into consideration the fact that the detection of various irregular obstacles in unstructured scenes plays a key role in trajectory planning, while the recognition of undulating road surface conditions in these scenes is crucial for speed planning. Therefore, we investigate unstructured scene understanding through 3D semantic occupancy prediction, which is used to detect irregular obstacles in unstructured scenes, and road surface elevation reconstruction, which characterizes the bumpy and uneven conditions of road surfaces. The dataset provides detailed annotations for 3D semantic occupancy prediction and road surface elevation reconstruction, offering a comprehensive representation of unstructured scenes. In addition, trajectory and speed planning information is provided to explore the relationship between perception and planning in unstructured scenes. Natural language descriptions of scenes are also provided to explore the interpretability of autonomous driving decision-making. Experiments have been conducted with various state-of-the-art methods to demonstrate the effectiveness of our dataset and the challenges posed by these tasks. To the best of our knowledge, this is the world's first comprehensive benchmark for perception in unstructured scenes, which serves as a valuable resource for extending autonomous driving technology from urban to unstructured scenes.