Anti-Noise 3D Object Detection of Multimodal Feature Attention Fusion Based on PV-RCNN

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• 3D object detection using camera and LiDAR fusion is hindered by sensor noise and feature space mismatches.
• This leads to feature information deviation and reduced detection performance.

Purpose of the Study:

• To develop a robust 3D object detection algorithm that overcomes environmental noise challenges.
• To improve feature fusion and detection accuracy in multimodal sensor data.

Main Methods:

• A point-guided feature abstraction method aggregates image and point cloud features to keypoints.
• A multimodal feature attention (MFA) mechanism adaptively fuses features from different sensor spaces.
• Projection-based farthest point sampling (P-FPS) optimizes point cloud downsampling for better keypoint projection.

Main Results:

• The proposed algorithm demonstrates improved detection performance and robustness, particularly in the presence of rain noise.
• Evaluations on three datasets and a physical test platform validate the algorithm's effectiveness.
• The method successfully addresses feature information deviation caused by sensor mismatches.

Conclusions:

• The developed 3D object detection approach enhances reliability in adverse environmental conditions.
• The combination of point-guided feature abstraction and MFA offers a significant advancement in sensor fusion for object detection.