Optimization-Based Post-Training Quantization With Bit-Split and Stitching.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) offer significant advancements but face storage and computational challenges.
• Network quantization is a key technique for improving DNN efficiency and hardware compatibility.
• Existing quantization methods often require extensive datasets and time-consuming fine-tuning to maintain accuracy.

Purpose of the Study:

• To address the limitations of current network quantization techniques.
• To develop a post-training quantization method that minimizes accuracy degradation without fine-tuning.
• To enable efficient low-bit quantization for deep neural networks.

Main Methods:

• Theoretical analysis of quantization loss, linking it to layer-wise activation reconstruction error.
• Development of an Optimization-based Post-training Quantization (OPQ) framework.
• Introduction of a novel Bit-split optimization approach within the OPQ framework.

Main Results:

• Demonstrated that quantization loss is bounded by activation reconstruction error.
• Achieved near-original model performance with 3-bit quantization.
• Validated the framework across diverse computer vision tasks (image classification, object detection, instance segmentation) and network architectures.

Conclusions:

• The proposed OPQ framework effectively reduces accuracy degradation during post-training quantization.
• Enables highly efficient 3-bit quantization without the need for fine-tuning.
• Offers a practical solution for deploying DNNs with reduced computational and storage overheads.