Dual Vision Transformer.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Self-attention mechanisms in Transformers are computationally intensive for high-resolution images.
• Existing methods often decompose attention over image patches, limiting holistic semantic understanding.
• Capturing global semantics while maintaining computational efficiency is a key challenge.

Purpose of the Study:

• To propose a novel Transformer architecture, Dual Vision Transformer (Dual-ViT), for efficient self-attention learning.
• To improve the capture of global semantics in image analysis.
• To enhance self-attention mechanisms for high-resolution inputs without substantial computational overhead.

Main Methods:

• Introduced Dual Vision Transformer (Dual-ViT) with parallel semantic and pixel pathways.
• Developed a semantic pathway to efficiently compress token vectors into global semantics.
• Integrated compressed global semantics as prior information for a pixel pathway to learn local details.
• Jointly trained both pathways to spread enhanced self-attention information.

Main Results:

• Dual-ViT effectively leverages global semantics to improve self-attention learning.
• The architecture achieves superior accuracy compared to state-of-the-art (SOTA) Transformer models.
• Comparable training complexity was maintained despite enhanced performance.

Conclusions:

• Dual-ViT offers an efficient and effective solution for self-attention in high-resolution image analysis.
• The dual-pathway approach successfully balances global semantic understanding and computational efficiency.
• This architecture represents a significant advancement in Transformer-based computer vision models.