Point Cloud Painting for 3D Object Detection with Camera and Automotive 3+1D RADAR Fusion.

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Advanced Driver-Assistance Systems (ADAS) utilize RADAR and cameras, but their integration in learning-based methods is underexplored.
• Existing methods often overlook the complementary strengths of RADAR and camera sensors for object detection.

Purpose of the Study:

• To propose a novel sensor fusion method for object detection in autonomous driving.
• To adapt PointPainting techniques for integrating 3+1D RADAR and camera semantic data.
• To improve detection accuracy by fusing geometrical and sequential sensor information.

Main Methods:

• A geometrical and sequential sensor fusion approach using 3+1D RADAR and camera data.
• Adaptation of PointPainting for RADAR point clouds, incorporating semantic information from camera instance segmentation (YOLOv8-seg).
• A heuristic error refinement stage followed by PointPillars for object detection on the painted RADAR point cloud.

Main Results:

• Significant improvement in object detection performance compared to a RADAR-only baseline.
• Mean Average Precision (mAP) increased from 41.18 to 52.67, a +27.9% enhancement.
• Validation on the View of Delft dataset, demonstrating effectiveness in urban driving scenarios.

Conclusions:

• The proposed sensor fusion method is effective for RADAR and camera integration in autonomous driving.
• This approach offers a substantial performance gain over using RADAR data alone.
• The technique successfully leverages semantic information from cameras to enhance RADAR-based object detection.