OmniHD-Scenes: A Next-Generation Multimodal Dataset for Autonomous Driving.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Deep learning for autonomous driving requires extensive, high-quality multimodal datasets.
• Current datasets often lack the comprehensive data coverage and detailed annotations needed for next-generation systems.
• Advanced sensors and diverse scene representation are critical for effective autonomous driving solutions.

Purpose of the Study:

• To introduce OmniHD-Scenes, a large-scale multimodal dataset for autonomous driving research.
• To provide comprehensive omnidirectional high-definition data from multiple advanced sensors.
• To establish benchmarks and evaluation metrics for 3D detection and semantic occupancy prediction.

Main Methods:

• The OmniHD-Scenes dataset integrates data from 128-beam LiDAR, six cameras, and six 4D imaging radar systems.
• A novel 4D annotation pipeline was developed, including precise 3D bounding boxes and semantic segmentation.
• An automated pipeline for dense occupancy ground truth generation was introduced, leveraging non-key frames.

Main Results:

• The dataset comprises 1501 clips with over 450K synchronized frames and 5.85M synchronized sensor data points.
• 200 clips are annotated with over 514K 3D bounding boxes and semantic segmentation.
• Experiments validate the effectiveness of low-cost sensor configurations and their robustness in adverse conditions.

Conclusions:

• OmniHD-Scenes provides a valuable resource for developing and testing autonomous driving algorithms.
• The dataset supports research into cost-effective sensor solutions using surround-view cameras and 4D imaging radar.
• The proposed benchmarks and metrics facilitate advancements in 3D detection and semantic occupancy prediction.