A Novel Low-Bit Quantization Strategy for Compressing Deep Neural Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Sophisticated neural network models increasingly demand significant memory and computational resources.
• This high demand hinders the deployment of neural networks on resource-constrained devices like ASICs, FPGAs, and mobile platforms.
• Efficient neural network compression and acceleration are crucial for broader application.

Purpose of the Study:

• To develop a novel strategy for training low-bit neural networks.
• To address the challenges of approximating activations and handling weight quantization in low-bit networks.
• To enable significant compression and acceleration of neural networks for mobile applications.

Main Methods:

• Implemented a strategy to train neural networks with weights and activations quantized to several bits.
• Approximated activations using low-bit discretization to reduce computational cost and memory.
• Specified a weight quantization and update mechanism to prevent gradient mismatch.
• Replaced full-precision operations with shift operations for quantized low-bit weights and activations.

Main Results:

• Successfully trained low-bit neural networks using the proposed quantization strategy.
• Demonstrated significant compression of neural networks.
• Achieved minimal loss in model accuracy despite aggressive quantization.
• Evaluated the method on common datasets, confirming its effectiveness.

Conclusions:

• The proposed method offers an effective approach to compress neural networks.
• Low-bit quantization of weights and activations significantly reduces computational and memory requirements.
• This strategy facilitates the deployment of advanced neural networks on mobile and edge devices with acceptable accuracy trade-offs.