NUPES : Non-Uniform Post-Training Quantization via Power Exponent Search | JoVE Visualize

Area of Science:

Artificial Intelligence
Machine Learning
Computer Engineering

Background:

Deep neural network (DNN) deployment is limited by high computational costs, especially for large language models (LLMs).
Quantization, converting floating-point to fixed-point representations, reduces memory and latency but uniform methods struggle with non-bell-shaped DNN weight/activation distributions and LLM outliers.
Existing post-training quantization techniques are insufficient for optimizing quantization parameters like exponents and weights effectively.

Purpose of the Study:

To propose NUPES, an advanced non-uniform quantization technique to overcome limitations of uniform quantization in DNNs and LLMs.
To develop a novel training paradigm for optimizing quantization operators and weights within the entire quantized space.
To enable integer-only, low-bit inference while maintaining model performance and achieving high compression rates.

Main Methods:

NUPES leverages power function-derived automorphisms to preserve scalar multiplications during quantization.
A new training paradigm learns quantized weights across the entire quantized space and optimizes the quantization operator's exponent parameter.
Numerical instabilities are alleviated, enabling end-to-end training of the quantization process.

Main Results:

NUPES achieves state-of-the-art compression rates in both data-free and data-driven quantization configurations.
The method effectively addresses the limitations of uniform quantization for distributions with outliers, particularly in transformers and LLMs.
Empirical benchmarks demonstrate NUPES's superior performance compared to previous post-training quantization techniques.

Conclusions:

NUPES offers a significant advancement in non-uniform quantization for efficient DNN and LLM deployment.
The proposed training paradigm successfully optimizes quantization parameters, leading to superior compression and performance.
NUPES provides a viable solution for deploying large models on resource-constrained hardware.