DeployFusion: A Deployable Monocular 3D Object Detection with Multi-Sensor Information Fusion in BEV for Edge...

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Existing multi-sensor fusion 3D object detection methods face challenges in remote detection accuracy and computational load.
• Need for efficient and accurate 3D object detection in real-time applications.

Purpose of the Study:

• To propose a novel Bird's-Eye View (BEV) based multi-sensor fusion 3D object detection method.
• To enhance detection accuracy for remote and small objects while reducing computational complexity.
• To enable real-time deployment on embedded platforms.

Main Methods:

• Utilized an enhanced lightweight EdgeNeXt feature extraction network with residual branches.
• Incorporated deformable convolution to expand receptive field and decrease computational complexity.
• Developed a two-stage feature fusion network for multi-sensor data alignment (image and point cloud).
• Employed a Transformer decoder for processing BEV feature sequences and emphasizing global spatial cues.

Main Results:

• Achieved a 4.5% improvement in NuScenes detection score compared to the baseline.
• Demonstrated a 5.5% increase in average precision for object detection.
• Real-time inference achieved with 138 ms per frame on a Jetson Orin NX embedded platform using TensorRT acceleration.

Conclusions:

• The proposed BEV-based method effectively addresses suboptimal remote detection and computational burden.
• The approach enables precise detection of distant small objects and real-time performance.
• Optimized model conversion and acceleration facilitate deployment on mobile devices for practical applications.