PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Multi-modal 3D object detection is crucial for autonomous driving in complex environments.
• Existing methods struggle with fusing LiDAR and camera data due to intrinsic discrepancies.
• Most multi-modal approaches underperform LiDAR-only methods.

Purpose of the Study:

• To propose PTA-Det, a novel method to enhance multi-modal 3D object detection performance.
• To address the challenges of cross-modal feature fusion in autonomous driving perception.

Main Methods:

• PTA-Det introduces a Pseudo Point Cloud Generation Network to represent image features as pseudo points.
• A transformer-based Point Fusion Transition (PFT) module deeply fuses LiDAR points and image-derived pseudo points.
• Features are fused in a unified point-based form for improved representation.

Main Results:

• PTA-Det effectively overcomes cross-modal feature fusion obstacles.
• The method achieves a complementary and discriminative representation for proposal generation.
• Experiments on the KITTI dataset show a 77.88% mean average precision (mAP) for car detection with sparse LiDAR data.

Conclusions:

• PTA-Det significantly improves multi-modal 3D object detection.
• The proposed method demonstrates the potential of integrating image-based pseudo points with LiDAR data.
• PTA-Det offers a promising solution for robust vehicle perception systems.