CAS-ViT: Convolutional Additive Self-Attention Vision Transformers for Efficient Mobile Applications.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Vision Transformers (ViTs) offer powerful global context but face limitations in efficiency due to complex operations.
• Resource-constrained environments like mobile devices require optimized neural network architectures for real-time applications.

Purpose of the Study:

• To introduce CAS-ViT (Convolutional Additive Self-attention Vision Transformers) for efficient deployment in mobile applications.
• To develop a novel Convolutional Additive Token Mixer (CATM) that reduces computational complexity while maintaining performance.

Main Methods:

• Proposed the Convolutional Additive Token Mixer (CATM) utilizing spatial and channel attention, eliminating matrix multiplication and Softmax.
• Introduced the Convolutional Additive Self-attention (CAS) block hybrid architecture incorporating CATM.
• Developed a family of lightweight CAS-ViT networks adaptable to various downstream tasks.

Main Results:

• CAS-ViT models (M and T) achieved 83.0%/84.1% top-1 accuracy on ImageNet-1K with only 12M/21M parameters.
• Demonstrated superior throughput on GPUs, ONNX, and iPhones compared to state-of-the-art backbones.
• Achieved a favorable balance between performance, efficient inference, and ease of deployment.

Conclusions:

• CAS-ViT offers an efficient and high-performing alternative to traditional Vision Transformers for mobile vision applications.
• The proposed CATM module effectively reduces computational overhead without sacrificing contextual understanding.
• CAS-ViT provides a versatile and deployable solution for diverse computer vision tasks on resource-limited devices.