BViT: Broad Attention-Based Vision Transformer.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Transformers, particularly Vision Transformers (ViT), leverage self-attention for image analysis.
• Existing ViT models often focus on attention within a single feature layer, neglecting cross-layer information complementarity.

Purpose of the Study:

• To introduce Broad Attention (BViT), a novel mechanism for Vision Transformers.
• To enhance ViT performance by incorporating attention relationships across different feature layers.

Main Methods:

• Broad attention integrates information through broad connections between transformer layers.
• Parameter-free attention mechanisms are employed to jointly utilize existing attention maps from different layers.
• The proposed BViT architecture is evaluated on image classification and object recognition tasks.

Main Results:

• BViT achieves superior top-1 accuracy on ImageNet (75.0% with 5M parameters, 81.6% with 22M parameters).
• On CIFAR10 and CIFAR100, BViT attains 98.9% and 89.9% accuracy, respectively, surpassing standard ViT with fewer parameters.
• Broad attention demonstrates generalization by improving performance in Swin Transformer, T2T-ViT, and LVT by over 1%.

Conclusions:

• Broad attention is a promising technique for improving the performance of attention-based models in computer vision.
• The parameter-free nature of broad attention offers an efficient way to boost ViT capabilities.
• BViT provides a strong baseline for future research in transformer-based vision models.