MuTrans: Multiple Transformers for Fusing Feature Pyramid on 2D and 3D Object Detection.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Feature pyramids are crucial for neural network perception tasks like object detection.
• Fusing multi-level and multi-sensor feature pyramids presents a significant challenge in object detection.

Purpose of the Study:

• To propose a novel framework, MuTrans (Multiple Transformers), for effective feature pyramid fusion.
• To enhance object detection accuracy in both 2D and 3D detectors, particularly for small objects.

Main Methods:

• MuTrans utilizes an encoder-decoder architecture with multiple Transformers to focus on significant features.
• The encoder incorporates Spatial-wise BoxAlign (SB) and Context-wise Affinity (CA) attention mechanisms.
• Low and High-level Fusion (LHF) and Pre-LN are employed to reduce computational complexity and accelerate training.

Main Results:

• MuTrans demonstrates superior detection accuracy compared to baseline methods, especially for small objects.
• Achieved a 2.1 higher APs index on MS-COCO 2017 and 2.18 higher 3D detection accuracy on KITTI for small objects.
• Showcased a 6.85 higher RC index on the CARLA urban driving simulator.

Conclusions:

• MuTrans offers a simple yet effective solution for feature pyramid fusion in object detection.
• The proposed attention mechanisms and fusion strategies lead to significant improvements in detection performance.
• MuTrans shows strong potential for real-world applications in autonomous driving and perception systems.