A LiDAR-Camera Joint Calibration Algorithm Based on Deep Learning.

Area of Science:

• Robotics
• Computer Vision
• Sensor Fusion

Background:

• Multisensor (MS) data fusion enhances vehicle environmental perception stability.
• Accurate MS joint calibration is crucial for effective sensor fusion.
• Traditional calibration methods are cumbersome and prone to significant errors due to manual feature extraction and registration.

Purpose of the Study:

• To propose a novel joint calibration algorithm for Light Laser Detection and Ranging (LiDAR) and camera systems.
• To develop a deep learning-based approach that eliminates the need for specialized calibration objects.
• To automatically determine the relative spatial positions of LiDAR and camera sensors.

Main Methods:

• A deep learning network model was constructed for automatic feature extraction, matching, and aggregation.
• The network comprises feature extraction (color and depth images), feature matching, and feature aggregation modules.
• A mathematical model was developed to analyze the sensor joint calibration process and determine rotation and translation matrix parameters.

Main Results:

• The proposed algorithm achieved an average translation error of 0.26 cm and an average rotation error of 0.02°.
• Performance was validated on the KITTI-odometry dataset, outperforming other advanced algorithms.
• The deep learning approach demonstrated high accuracy in determining relative spatial positions between LiDAR and camera.

Conclusions:

• The developed deep learning algorithm offers an efficient and accurate solution for joint LiDAR-camera calibration.
• This method significantly reduces calibration complexity and errors compared to traditional techniques.
• The approach holds promise for improving the stability and reliability of vehicle environmental perception systems.