QuIP: 2-Bit Quantization of Large Language Models With Guarantees.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Post-training parameter quantization is crucial for deploying large language models (LLMs) efficiently.
• Existing quantization methods face challenges in maintaining model performance at very low bit precision.

Purpose of the Study:

• To introduce a novel quantization method, Quantization with Incoherence Processing (QuIP), for LLMs.
• To theoretically analyze LLM-scale quantization algorithms and demonstrate the benefits of weight and Hessian incoherence.

Main Methods:

• QuIP employs an adaptive rounding procedure to minimize a quadratic proxy objective.
• It utilizes pre- and post-processing with random orthogonal matrices to induce weight and Hessian incoherence.
• Theoretical analysis is provided for QuIP and extended to the OPTQ algorithm.

Main Results:

• Incoherence preprocessing demonstrably improves existing quantization algorithms.
• QuIP achieves the first viable LLM quantization using only two bits per weight.
• The theoretical framework developed is applicable to other quantization methods like OPTQ.

Conclusions:

• Weight and Hessian incoherence is a beneficial property for LLM quantization.
• QuIP offers a significant advancement in efficient LLM deployment.
• The proposed method and theoretical analysis pave the way for further research in low-bit LLM quantization.