LB-PTQ: Effective Low-Bit Post-Training Quantization for Vision Transformers.

Area of Science:

• Computer Vision
• Deep Learning
• Model Compression

Background:

• Vision Transformers (ViTs) achieve state-of-the-art results in computer vision but have high computational complexity and large parameter counts.
• Deploying large ViT models on resource-constrained devices like mobile phones is challenging due to their size and computational demands.

Purpose of the Study:

• To develop a novel post-training quantization approach for Vision Transformers (ViTs) that reduces model size without requiring re-training.
• To overcome the sub-optimal results of layer-wise quantization by proposing a unified framework for joint layer optimization.

Main Methods:

• A novel post-training quantization approach is proposed, optimizing all layers jointly to minimize overall network output error.
• The additivity property of ViTs is leveraged for efficient joint optimization, resulting in a linear time complexity algorithm.
• Extensive experiments were conducted on the ImageNet dataset to validate the effectiveness of the proposed method.

Main Results:

• The proposed method achieves state-of-the-art improvements on various ViT models, reducing bit width to 6-bit without accuracy loss.
• At 4-bit precision, the approach shows significant outperformance over existing methods on ViTS, ViT-B, DeiT-S, and DeiT-B models.
• Quantized models demonstrate 1.5x to 1.7x inference speedups on NVIDIA A100 GPUs, indicating practical hardware deployment benefits.

Conclusions:

• The novel joint optimization framework effectively quantizes ViTs to very low bit widths, maintaining high accuracy.
• This approach significantly enhances the efficiency of Vision Transformers, making them suitable for deployment on mobile and edge devices.
• The method offers a practical solution for compressing large ViT models, improving inference speed and reducing computational costs.