S2AFormer: Strip Self-Attention for Efficient Vision Transformer.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Vision Transformers (ViT) excel at capturing global dependencies but suffer from quadratic computational complexity.
• Existing hybrid models struggle with the high cost of pairwise token interactions in self-attention.
• Efficient deployment of advanced computer vision models remains a challenge.

Purpose of the Study:

• To develop an efficient Vision Transformer architecture that overcomes the computational limitations of standard self-attention.
• To introduce a novel Strip Self-Attention (SSA) mechanism for improved performance-efficiency trade-off.
• To enhance the fusion of local and global feature extraction in computer vision models.

Main Methods:

• Developed S2AFormer, an efficient Vision Transformer architecture.
• Introduced the Strip Self-Attention (SSA) mechanism with joint spatial-and-channel compression of query, key, and value tensors.
• Integrated Hybrid Perception Blocks (HPBs) to combine CNN local inductive biases with Transformer global modeling.

Main Results:

• S2AFormer demonstrates substantial accuracy improvements across image classification, semantic segmentation, and object detection tasks.
• The model achieves superior inference speed and throughput on both GPU and non-GPU platforms.
• Achieved an excellent balance between accuracy and computational efficiency.

Conclusions:

• S2AFormer offers a highly competitive and efficient solution for Vision Transformer architectures.
• The Strip Self-Attention mechanism effectively reduces computational cost while maintaining representational power.
• The proposed architecture is suitable for practical deployment in various computer vision applications.