Unsupervised Optical-Sensor Extrinsic Calibration via Dual-Transformer Alignment.

Area of Science:

• Robotics
• Computer Vision
• Sensor Fusion

Background:

• Accurate extrinsic calibration between optical sensors (camera, LiDAR) is vital for multimodal perception.
• Traditional calibration methods struggle with robustness in challenging optical conditions (glare, low light) and require manual operations.

Purpose of the Study:

• To propose a fully unsupervised, end-to-end framework for robust camera-LiDAR extrinsic calibration.
• To overcome limitations of traditional methods in complex environments and manual intervention.

Main Methods:

• A dual-Transformer architecture utilizing Vision Transformer (ViT) for image semantics and Point Transformer for 3D point cloud geometry.
• Cross-modal feature alignment and fusion via a neural network, followed by regression for the 6-DoF extrinsic transformation matrix.
• A multi-constraint loss function to enhance inter-modal structural consistency.

Main Results:

• Achieved a mean rotation error of 0.21° and translation error of 3.31 cm on the KITTI benchmark.
• Attained an average reprojection error of 1.52 pixels on a self-collected dataset.
• Demonstrated improved calibration stability and accuracy compared to traditional approaches.

Conclusions:

• The proposed framework offers a generalizable and robust solution for optical-sensor extrinsic calibration.
• Enables precise and self-sufficient perception in real-world applications without manual calibration targets.
• Highlights the potential of unsupervised, end-to-end learning for sensor fusion tasks.