The ApolloScape Open Dataset for Autonomous Driving and Its Application

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Autonomous driving relies on robust perception models for tasks like 3D mapping and object understanding.
• Developing these models is hindered by the lack of large-scale, richly labeled datasets for training and evaluation.
• Existing datasets like KITTI and Cityscapes have limitations in scale and annotation richness.

Purpose of the Study:

• To introduce the ApolloScape dataset, a large-scale, richly annotated resource for autonomous driving research.
• To present a sensor fusion scheme integrating diverse data sources for enhanced self-localization and semantic segmentation.
• To demonstrate the benefits of joint multi-task learning using the ApolloScape dataset.

Main Methods:

• Development of the ApolloScape dataset with extensive annotations including dense point clouds, semantic labeling, and instance segmentation.
• Creation of specialized tools and algorithms (e.g., joint 3D-2D segment labeling) to accelerate data annotation.
• Implementation of a sensor fusion scheme combining camera, GPS/IMU, and 3D semantic maps.
• Joint learning and inference of multiple perception tasks for autonomous driving.

Main Results:

• ApolloScape provides significantly larger and richer labeled data (at least 15x more images per task) compared to state-of-the-art datasets.
• The proposed sensor fusion scheme achieves robust self-localization and semantic segmentation.
• Joint learning of multiple tasks and sensor fusion demonstrably improve system robustness and accuracy.

Conclusions:

• The ApolloScape dataset is a valuable resource for advancing autonomous driving research.
• Sensor fusion and joint multi-task learning are crucial for developing more robust and accurate autonomous driving systems.
• The dataset and methodologies are expected to motivate further research in multi-sensor fusion and multi-task learning in computer vision.