Convolutional Neural Networks Quantization with Double-Stage Squeeze-and-Threshold.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Deep Convolutional Neural Networks (DCNNs) offer efficiency gains through low-precision inference.
• Neural network quantization, while reducing memory and power, typically leads to accuracy degradation.

Purpose of the Study:

• To propose a novel quantization method, double-stage Squeeze-and-Threshold (ST), to mitigate accuracy loss in DCNNs.
• To enable efficient inference without compromising model performance.

Main Methods:

• Utilizes an attention mechanism to adjust activations and learn thresholds for feature distinction.
• Divides numerically rich activations into intervals using learned thresholds.
• Supports both binarization and multi-bit quantization.

Main Results:

• Achieved state-of-the-art results in both binarization and multi-bit quantization.
• Outperformed previous SOTA by 0.2% in binarization (ReActNet).
• Exceeded full-precision baseline by 0.4% in 3-bit ResNet-18 accuracy.

Conclusions:

• The double-stage ST method effectively bridges the accuracy gap between quantized and full-precision models.
• The method is easily integrated before convolution and incurs no inference cost.
• Demonstrates the potential of low-precision DCNNs for efficient and accurate image recognition.