TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Sensor fusion is crucial for autonomous driving perception.
• Existing geometry-based fusion methods struggle with complex, dynamic traffic scenarios in end-to-end driving.
• Imitation learning with current sensor fusion underperforms in dense agent environments.

Purpose of the Study:

• To propose a novel mechanism, TransFuser, for integrating image and LiDAR representations for enhanced autonomous driving.
• To leverage self-attention mechanisms for effective multi-modal sensor fusion.
• To improve the performance of end-to-end driving models in challenging conditions.

Main Methods:

• Developed TransFuser, a self-attention-based mechanism for fusing image and LiDAR data.
• Utilized transformer modules at multiple resolutions to fuse perspective and bird's eye view feature maps.
• Experimentally validated the approach on a new benchmark and the CARLA urban driving simulator leaderboard.

Main Results:

• TransFuser significantly outperforms prior work on the CARLA leaderboard, achieving a higher driving score.
• Reduced average collisions per kilometer by 48% compared to geometry-based fusion.
• Demonstrated superior performance in complex driving scenarios with dense dynamic agents.

Conclusions:

• TransFuser offers a more effective approach to sensor fusion for autonomous driving than traditional methods.
• Self-attention-based fusion is a promising direction for improving end-to-end driving systems.
• The proposed method enhances safety and performance in challenging traffic conditions.