Surrounding-aware representation prediction in Birds-Eye-View using transformers.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Birds-Eye-View (BEV) maps are crucial for autonomous systems, representing environmental sensory data.
• Generating semantic BEV maps involves object detection and segmentation, tasks often limited by current Convolutional Neural Network (CNN) capacities.
• Existing CNNs struggle with multi-scale and multi-class elements, hindering accurate representation prediction.

Purpose of the Study:

• To develop advanced neural networks for accurate Birds-Eye-View (BEV) semantic representation prediction.
• To overcome the limitations of CNNs in perceiving subtle environmental nuances and handling complex scene elements.
• To propose a novel, end-to-end forecasting approach for BEV map generation using Transformers.

Main Methods:

• Proposed novel neural networks utilizing Transformers exclusively, avoiding convolutional layers.
• Developed a new Transformer-powered pixel generation method for BEV maps.
• Introduced a novel algorithm for image-to-BEV transformation and an attention-based network for image feature extraction.

Main Results:

• The proposed Transformer-based models demonstrated superior performance compared to state-of-the-art CNNs.
• Achieved a relative improvement of 2.4% on the NuScenes dataset and 5.2% on the Argoverse 3D dataset.
• Successfully generated BEV maps with per-class probabilities in an end-to-end forecasting manner.

Conclusions:

• The novel Transformer architecture offers a more efficient and accurate solution for BEV semantic representation prediction.
• This approach effectively addresses the limitations of CNNs in handling multi-scale and multi-class environmental data.
• The proposed method represents a significant advancement in semantic BEV map generation for applications like autonomous driving.