LiDAR-Free 3D Auto-Labeling via Radar-Visual Spatio-Temporal Consistency

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Vision foundation models (VFMs) generate 2D instance masks but struggle with 3D annotation due to scale ambiguity and noise.
• Existing 3D auto-labeling methods often require expensive LiDAR sensors or lack physical plausibility in dynamic scenes.

Purpose of the Study:

• To develop a LiDAR-free auto-labeling framework for 3D annotation using radar and visual data.
• To enhance 3D geometry accuracy by leveraging cross-modal spatio-temporal consistency.

Main Methods:

• Associating radar points, 2D masks, and pseudo-point clouds into object-centric sequences.
• Employing an uncertainty-aware pose fusion module with automatically solved road priors.
• Refining pseudo-point clouds by optimizing semantic landmarks from temporally consistent masks.

Main Results:

• Achieved 49.1% bird's-eye-view (BEV) IoU and 43.0% 3D IoU on a real-world roadside dataset.
• Outperformed a radar-camera fusion baseline by 5.5/5.9 points in BEV and 3D IoU.
• Demonstrated the utility of generated pseudo-labels and semantic enhancement for downstream tasks.

Conclusions:

• The proposed radar-visual framework effectively corrects 3D geometry for auto-labeling without LiDAR.
• The method shows strong performance and potential for improving 3D annotation in dynamic roadside environments.
• Further validation across diverse configurations is recommended for broader applicability.