LiDAR-360 RGB Camera-360 Thermal Camera Targetless Calibration for Dynamic Situations

Area of Science:

• Robotics and Autonomous Systems
• Computer Vision
• Sensor Fusion

Background:

• Autonomous systems increasingly integrate multiple sensors like RGB cameras, thermal cameras, and LiDAR for comprehensive environmental data.
• Current research often neglects the combined potential of all three sensor types, focusing on pairs.
• Ego-motion compensation in dynamic environments presents challenges due to sensor characteristics like LiDAR's rotation and camera blind spots.

Purpose of the Study:

• To propose a novel method for simultaneous registration of LiDAR, panoramic RGB, and panoramic thermal cameras.
• To address ego-motion compensation challenges in dynamic environments without calibration targets.
• To enhance the accuracy and reliability of multi-sensor calibration for autonomous systems.

Main Methods:

• Feature extraction from RGB images, thermal data, and LiDAR point clouds using a novel technique.
• Ego-motion compensation based on extracted features to optimize the initial dataset.
• Refinement of raw features for enhanced calibration accuracy and precise alignment.

Main Results:

• Demonstrated effectiveness in enhancing multi-sensor calibration compared to existing methods.
• Achieved up to 30% improvement in LiDAR and camera calibration accuracy at speeds around 9 m/s.
• Validated the proposed method's capability in dynamic and challenging environmental conditions.

Conclusions:

• The novel method enables simultaneous registration of LiDAR, panoramic RGB, and thermal cameras in dynamic settings.
• The approach effectively handles ego-motion compensation, improving calibration accuracy and reliability.
• This work has the potential to significantly advance the performance of autonomous systems in real-world applications.