Graph Enhanced Multi-Modal Network of 4-D Radar-Camera Fusion for Perception in Autonomous Systems

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Autonomous systems utilize diverse sensors (vision, radar, LiDAR) but face individual failure modes.
• LiDAR-vision fusion is common, but 4D radar and monocular optics synergy is underexplored.

Purpose of the Study:

• To develop a graph-enhanced multi-modal architecture for scene-level 3D perception using 4D radar and monocular camera data.
• To address the gap in understanding the synergistic potential of 4D radar and monocular optics.

Main Methods:

• An image-guided point densification scheme (SAA) augments sparse radar data with camera information.
• A pose-invariant cross-modal fusion layer integrates radar and image features.
• A dynamic hypergraph assembly and HyperGCN module capture inter-object dependencies for 3D perception.

Main Results:

• The system achieves an mAP of 69.3 and mAOS of 59.8 on the View-of-Delft (VOD) dataset.
• Reliable perception is demonstrated across diverse environmental conditions by combining radar velocity and camera appearance data.
• Ablation studies quantify the impact of geometric invariance on detection.

Conclusions:

• The proposed graph-enhanced multi-modal architecture effectively leverages 4D radar and monocular camera data for robust 3D scene perception.
• This approach enhances autonomous system reliability by overcoming limitations of individual sensors.
• The study highlights the significant potential of fusing 4D radar with monocular vision for advanced perception tasks.