Targetless LiDAR-camera extrinsic calibration via semantic distribution alignment

LiDAR-camera fusion systems are widely used in robotic localization and perception, where accurate extrinsic calibration is crucial for multi-sensor fusion. During long-term operation, extrinsic parameters can drift due to vibration and other disturbances, while target-based recalibration is inconvenient in the field and targetless approaches often suffer from highly non-convex objectives and limited robustness in challenging outdoor scenes.

We propose a targetless LiDAR-camera extrinsic calibration method by minimizing a semantic distribution consistency risk on SE(3). We align semantic probability distributions from the two sensing modalities in the image domain and freeze the pixel sampling measure at an anchor pose, so that pixel weighting no longer depends on the current extrinsic estimate and the objective landscape remains stable during optimization. On top of this anchor-fixed measure, we introduce a direction-aware weighting strategy that emphasizes pixels sensitive to yaw perturbations, improving the conditioning of rotation estimation. We further use a globally balanced Jensen-Shannon divergence to mitigate semantic class imbalance and enhance robustness.

Experiments on the KITTI Odometry dataset show that the proposed method reliably converges from substantial initial perturbations and yields stable extrinsic estimates.

The results indicate that the method is promising for maintaining long-term LiDAR-camera calibration in real-world robotic systems.