Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Area of Science:

• Robotics
• Computer Vision
• Sensor Fusion

Background:

• Place recognition is essential for autonomous systems, enabling both mapping and localization.
• Current single-sensor approaches struggle with reliability in adverse environmental conditions.
• Integrating data from multiple sensor types (heterogeneous measurements) offers a promising solution.

Purpose of the Study:

• To propose a novel framework for long-term place recognition using heterogeneous sensor data.
• To enable retrieval of radar scans from existing Light Detection and Ranging (Lidar) maps.
• To develop a method that performs place recognition across different sensor modalities (radar and Lidar).

Main Methods:

• A deep neural network architecture was developed for joint training of radar and Lidar data.
• Shared embeddings for radar and Lidar data were extracted during the testing phase.
• The framework was validated through extensive tests and generalization experiments on public datasets.

Main Results:

• The proposed model successfully performs multi-modal place recognition: Lidar-to-Lidar (L2L), Radar-to-Radar (R2R), and Radar-to-Lidar (R2L).
• The learned model requires only a single training instance to achieve cross-modal recognition capabilities.
• Experimental results demonstrate the effectiveness and generalization of the heterogeneous measurement framework.

Conclusions:

• The developed heterogeneous measurement framework significantly improves place recognition robustness, especially in adverse conditions.
• The joint learning of shared embeddings allows for effective cross-modal place recognition (e.g., radar queries on Lidar maps).
• The study provides a valuable contribution to sensor fusion for autonomous navigation and mapping.