Advancing Vision Transformer with Enhanced Spatial Priors.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Vision Transformer (ViT) models are popular but lack spatial priors and have quadratic computational complexity.
• Existing methods like RMT introduce spatial priors but have limitations.
• Addressing limitations in ViT's self-attention mechanism is crucial for broader applicability.

Purpose of the Study:

• To introduce the Euclidean enhanced Vision Transformer (EVT), a novel vision backbone.
• To enhance spatial information modeling and improve computational efficiency over previous models.
• To demonstrate EVT's effectiveness in various computer vision applications.

Main Methods:

• EVT incorporates Euclidean distance decay for improved spatial information modeling.
• It utilizes a spatially-independent grouping approach, replacing RMT's decomposed attention.
• The model is evaluated on Image Classification, Object Detection, Instance Segmentation, and Semantic Segmentation tasks.

Main Results:

• EVT achieves 86.6% top1-accuracy on ImageNet-1k without additional training data.
• The model demonstrates exceptional performance across diverse computer vision benchmarks.
• EVT effectively integrates spatial priors into the self-attention mechanism.

Conclusions:

• EVT offers a more sophisticated and adaptable approach to incorporating spatial priors in vision transformers.
• The proposed methods overcome limitations of previous models like RMT.
• EVT shows significant potential for advancing computer vision research and applications.