Exploiting Retraining-Based Mixed-Precision Quantization for Low-Cost DNN Accelerator Design.

Area of Science:

• Computer Engineering
• Artificial Intelligence
• Hardware Acceleration

Background:

• Deep neural networks (DNNs) require significant computational resources, limiting their deployment on edge devices.
• Retraining-based quantization is a common technique to reduce memory access and model size for DNNs.
• Uniform quantization to 4-bit can maintain accuracy but may not be optimal for all network layers.

Purpose of the Study:

• To propose a retraining-based mixed-precision quantization approach for enhanced energy efficiency in DNNs.
• To develop a customized DNN accelerator capable of handling variable bit widths.
• To evaluate the compression ratio and energy savings of the proposed method and accelerator.

Main Methods:

• Implemented a mixed-precision quantization strategy by assigning extra bits to weights with high switching frequency during retraining.
• Mitigated gradient noise by adjusting the learning rate near quantization thresholds.
• Designed and fabricated a customized DNN accelerator with dynamically reconfigurable processing elements (PEs) supporting 2-4 bit precision.

Main Results:

• The proposed mixed-precision quantized network (MPQ-network) achieved a 1.37x better compression ratio for VGG-9 on CIFAR-10 compared to uniform 4-bit quantization.
• The customized accelerator demonstrated 1.29x energy savings for VGG-9 on CIFAR-10 compared to state-of-the-art accelerators.
• Full precision classification accuracy was maintained with the proposed quantization method.

Conclusions:

• The proposed mixed-precision quantization and customized accelerator effectively improve compression and energy efficiency for DNNs on resource-constrained devices.
• The dynamic reconfigurability of the accelerator allows for flexible processing of variable bit widths.
• This approach offers a promising solution for deploying complex DNN models in power-sensitive applications.