Deployable mixed-precision quantization with co-learning and one-time search.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) require significant computational resources, limiting their deployment on edge devices.
• Mixed-precision quantization is crucial for reducing DNN model size and inference latency.
• Optimizing bit-width allocation for different layers in mixed-precision quantization remains a significant challenge.

Purpose of the Study:

• To introduce Cobits, an efficient and effective framework for deployable mixed-precision quantization.
• To address the challenge of optimal bit-width configuration for DNNs in resource-constrained environments.
• To develop a method for dynamically adapting quantization parameters and generalizing to various backends.

Main Methods:

• Cobits utilizes the relationship between real-valued input ranges and quantized ranges to assign bit-widths.
• A co-learning approach entangles and learns quantization parameters, differentiating shared and specific parts.
• Normal quantizers are upgraded to dynamic quantizers to mitigate statistical issues in mixed-precision supernets.
• Quantized real-valued ranges are used to derive bit-sensitivity for efficient bit-width configuration without iterative validation.

Main Results:

• Cobits outperforms state-of-the-art quantization methods on ImageNet and COCO datasets.
• The framework demonstrates superior efficiency in mixed-precision quantization.
• Cobits dynamically adapts to varying bit-widths and generalizes to different deployable backends.
• The proposed method eliminates the need for iterative validation dataset evaluations for bit-width determination.

Conclusions:

• Cobits provides an efficient and effective solution for deployable mixed-precision quantization.
• The framework's intelligent bit-width assignment and co-learning approach enhance DNN performance and efficiency.
• Cobits offers a generalizable and adaptive solution for deploying DNNs in resource-constrained settings.