PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Transformer architectures, particularly those utilizing self-attention, have surpassed Convolutional Neural Networks (CNNs) in various computer vision applications.
• Self-attention mechanisms allow transformers to capture dependencies across both short and long distances, creating extensive receptive fields.

Purpose of the Study:

• To propose the Parallel Local-Global Vision Transformer (PLG-ViT), a versatile backbone model designed to enhance computer vision performance.
• To effectively and efficiently represent short- and long-range spatial interactions by merging local and global self-attention features.

Main Methods:

• Developed the Parallel Local-Global Vision Transformer (PLG-ViT) by integrating local window self-attention with global self-attention.
• Evaluated PLG-ViT on image classification, object detection, instance segmentation, and semantic segmentation tasks.
• Compared PLG-ViT against CNN-based and state-of-the-art transformer-based architectures, including ConvNeXt and Swin Transformer.

Main Results:

• PLG-ViT demonstrated superior performance over CNN and existing transformer models in various computer vision tasks.
• Achieved high Top-1 accuracy on ImageNet-1K: 83.4% (27M parameters), 84.0% (52M parameters), and 84.5% (91M parameters).
• Outperformed similarly sized networks like ConvNeXt and Swin Transformer.

Conclusions:

• The proposed PLG-ViT effectively fuses local and global self-attention for efficient and powerful visual representation.
• PLG-ViT offers a competitive and efficient alternative to current leading computer vision backbones.
• The model shows significant potential for complex downstream tasks in computer vision.