LRQuant+: A Unified and Learnable Framework to Post-Training Quantization for Transformer-Based Large Foundation...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Post-training quantization (PTQ) accelerates inference and reduces memory for large foundation models (LFMs) without retraining.
• Existing PTQ methods struggle with hand-crafted scaling factors, ignore directional quantization errors, and lack broad applicability.
• Current quantization error metrics (e.g., L2-norm) do not capture directional shifts, leading to suboptimal performance.

Purpose of the Study:

• To develop a unified, learnable, and robust post-training quantization framework (LRQuant) for transformer-based LFMs.
• To address limitations of existing PTQ methods by introducing learnable scaling factors and a novel loss function.
• To provide a comprehensive evaluation across diverse LFMs and quantization scenarios, including challenging low-bit settings.

Main Methods:

• Introduced a block-wise learnable paradigm for optimal scaling factor determination, initialized with logarithmic activation equivalents.
• Proposed a novel Negative Logarithm of Cosine Similarity (NLC) loss to better capture quantization errors beyond MSE.
• Developed LRQuant+ with a dynamic loss weighting scheme, learnable rotation vectors, and a two-branch optimization for error propagation and reconstruction.

Main Results:

• LRQuant and LRQuant+ demonstrate superior performance across various LFMs, including LLMs, ViTS, and MLLMs.
• The methods achieve effectiveness in both weight-activation and weight-only quantization, particularly in challenging W4A4 and W2A16 scenarios.
• Experimental results validate the unified applicability and robustness of the proposed LRQuant framework.

Conclusions:

• LRQuant offers a significant advancement in post-training quantization for LFMs, improving efficiency and accuracy.
• The learnable approach and novel NLC loss effectively mitigate quantization errors and enhance model robustness.
• The framework's versatility across different models and quantization bit-widths makes it a valuable tool for deploying LFMs.