V2X-ViTv2: Improved Vision Transformers for Vehicle-to-Everything Cooperative Perception

Area of Science:

• Autonomous Systems
• Computer Vision
• Communication Engineering

Background:

• Autonomous vehicles require robust perception systems for safe operation.
• Current perception systems face limitations in complex and dynamic environments.
• Vehicle-to-Everything (V2X) communication offers potential for cooperative perception.

Purpose of the Study:

• To develop a cooperative perception framework leveraging V2X communication for autonomous vehicles.
• To introduce novel Vision Transformer (ViT) models tailored for V2X-based perception.
• To enhance the accuracy and robustness of 3D object detection in autonomous driving.

Main Methods:

• Proposed V2X-ViTv1 and V2X-ViTv2 architectures utilizing holistic attention and multi-scale window self-attention modules.
• Developed a unified Transformer architecture to address V2X challenges like asynchronous data and pose errors.
• Employed advanced data augmentation techniques specific to V2X applications.
• Utilized CARLA and OpenCDA simulators to create a large-scale V2X perception dataset.

Main Results:

• V2X-ViTs demonstrated state-of-the-art performance in 3D object detection.
• The framework showed significant robustness in harsh and noisy real-world and synthetic datasets.
• Effective fusion of information across multiple on-road agents was achieved.

Conclusions:

• V2X communication integrated with novel Transformer models significantly enhances autonomous vehicle perception.
• The proposed V2X-ViTs framework offers a robust solution for 3D object detection.
• This approach holds promise for improving the safety and reliability of autonomous driving systems.