PT-BitNet: Scaling up the 1-Bit large language model with post-training quantization.

Area of Science:

• Artificial Intelligence
• Computer Science
• Machine Learning

Background:

• Large Language Models (LLMs) face hardware and cost constraints.
• Quantization techniques offer a solution to reduce LLM resource requirements.
• BitNet demonstrated the efficacy of ternary quantization but requires training from scratch.

Purpose of the Study:

• Introduce PT-BitNet, a post-training quantization method for LLMs.
• Extend BitNet's ternary quantization benefits to large-scale models (up to 70B parameters).
• Reduce LLM size and inference latency with minimal performance degradation.

Main Methods:

• Developed a two-stage post-training quantization algorithm.
• Stage 1: Transform weight distribution for quantization compatibility.
• Stage 2: Optimize quantized weights in a block-wise manner.

Main Results:

• PT-BitNet successfully quantizes LLMs up to 70B parameters.
• Achieved substantial reductions in model size and inference time.
• Maintained high task performance, e.g., 61% accuracy on downstream tasks for a 70B model.

Conclusions:

• PT-BitNet effectively scales ternary quantization to large LLMs.
• Outperforms previous methods like BitNet b.158 (51.2% accuracy) on large models.
• Provides a viable solution for deploying efficient and performant LLMs.