A survey of low-bit large language models: Basics, systems, and algorithms

Area of Science:

• Artificial Intelligence
• Computer Science
• Machine Learning

Background:

• Large language models (LLMs) demonstrate advanced natural language processing capabilities.
• High memory and computational demands hinder the practical deployment of LLMs.

Purpose of the Study:

• To provide a comprehensive survey of low-bit quantization methods for LLMs.
• To analyze fundamental principles, system implementations, and algorithmic strategies.
• To offer insights for enhancing LLM efficiency and applicability.

Main Methods:

• Review of basic concepts and data formats for low-bit LLMs.
• Analysis of frameworks and systems for hardware deployment.
• Categorization of techniques and toolkits for efficient training and inference.

Main Results:

• Low-bit quantization reduces memory usage and computational requirements for LLMs.
• Various methods, systems, and algorithms facilitate efficient low-bit LLM deployment.
• The survey systematically covers basic, system, and algorithmic perspectives.

Conclusions:

• Low-bit quantization is crucial for mitigating LLM resource constraints.
• This survey provides valuable guidelines for future research in efficient LLMs.
• Advancements in low-bit quantization will enhance LLM applicability across diverse platforms.