End-to-End Multimodal Sensor Dataset Collection Framework for Autonomous Vehicles

Area of Science:

• Robotics and Autonomous Systems
• Computer Vision
• Sensor Technology

Background:

• Autonomous vehicles require robust perception using multiple redundant sensors like cameras, LiDAR, and radar.
• Sensor calibration and synchronization are critical for multi-sensor systems, impacting object detection and path planning.
• Existing research often lacks comprehensive fusion of camera, LiDAR, and radar data and a scalable implementation.

Purpose of the Study:

• To address research gaps in multi-sensor fusion and scalable implementation for autonomous driving.
• To introduce a generic, end-to-end framework for sensor dataset collection and fusion.
• To develop a universal toolbox for calibrating and synchronizing diverse sensors.

Main Methods:

• Developed an end-to-end framework integrating hardware solutions and sensor fusion algorithms.
• Incorporated a diverse set of sensors: camera, LiDAR, and radar.
• Created a universal toolbox for sensor calibration and synchronization based on sensor characteristics.

Main Results:

• Successfully integrated camera, LiDAR, and radar sensors within the framework prototype.
• Presented fusion algorithms that leverage the strengths of each sensor for object detection and tracking.
• Demonstrated the framework's generality for various robotic and autonomous applications.

Conclusions:

• The proposed framework provides a generalized, scalable, and user-friendly solution for multi-sensor perception.
• It effectively addresses the need for fused and synchronized data from cameras, LiDAR, and radar.
• The framework facilitates quick and large-scale deployment in autonomous systems.