Multi-tailed vision transformer for efficient inference.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Vision Transformer (ViT) models excel in image recognition but face computational challenges due to quadratic complexity with input tokens.
• Existing methods prune tokens within the encoder, leaving initial tokenization inefficient.

Purpose of the Study:

• To propose an efficient Vision Transformer architecture that directly reduces computational cost by optimizing input token sequences.
• To introduce a novel Multi-Tailed Vision Transformer (MT-ViT) that balances accuracy and computational efficiency.

Main Methods:

• Developed a Multi-Tailed Vision Transformer (MT-ViT) with multiple tails generating visual sequences of varying lengths.
• Integrated a tail predictor, optimized end-to-end using the Gumbel-Softmax trick, to select the most efficient tail for prediction.
• Evaluated MT-ViT on the ImageNet-1K dataset.

Main Results:

• MT-ViT achieved significant reduction in Floating Point Operations (FLOPs).
• The model demonstrated no degradation in accuracy compared to standard Vision Transformers.
• MT-ViT outperformed existing methods in terms of both accuracy and computational efficiency.

Conclusions:

• The proposed MT-ViT effectively reduces computational complexity in Vision Transformers by optimizing input tokenization.
• MT-ViT offers a promising direction for developing more efficient and accurate vision models.
• This approach provides a practical solution for deploying Vision Transformers in resource-constrained environments.