Pushing the Limit of Post-Training Quantization.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Post-training quantization (PTQ) is a cost-effective method for creating efficient low-precision neural networks.
• Existing PTQ methods struggle with extremely low-bit quantization (e.g., INT2) due to weight perturbation and unaddressed activation quantization.

Purpose of the Study:

• To theoretically analyze the failure of current PTQ methods in extremely low-bit settings.
• To develop a novel PTQ framework capable of pushing quantization limits to INT2.
• To extend the framework for mixed-precision quantization.

Main Methods:

• Developed a unified theoretical analysis to understand PTQ limitations at extremely low bit-widths.
• Proposed Brecq and QDrop techniques to address weight perturbation and activation quantization issues.
• Constructed the Q-Limit framework, further extended for mixed-precision quantization.

Main Results:

• The Q-Limit framework successfully enables post-training quantization down to INT2 precision.
• Achieved performance comparable to quantization-aware training (QAT) for models like ResNet and MobileNetV2.
• Established a new state-of-the-art for post-training quantization in visual recognition, detection, and language processing tasks.

Conclusions:

• The proposed Q-Limit framework overcomes critical challenges in extremely low-bit PTQ.
• This work demonstrates the feasibility of achieving high performance with PTQ at INT2 precision.
• The framework offers a new state-of-the-art for efficient neural network quantization without extensive retraining.