IVS-Caffe-Hardware-Oriented Neural Network Model Development.

Area of Science:

• Computer Science
• Artificial Intelligence
• Hardware Acceleration

Background:

• Convolutional Neural Networks (CNNs) are computationally intensive.
• Quantization is crucial for deploying CNNs on hardware accelerators with limited resources.
• Existing quantization methods often lead to accuracy degradation.

Purpose of the Study:

• To develop a hardware-oriented tool, IVS-Caffe, for simulating and training quantized CNNs.
• To address the accuracy drop caused by bit-accurate quantization in multipliers and accumulators.
• To enable accurate analysis of CNN performance on hardware accelerators at various bit widths.

Main Methods:

• IVS-Caffe simulates hardware behavior of CNN inference, including quantized weights, inputs, and outputs.
• It quantizes multipliers and accumulators to achieve bit-accurate results.
• A novel algorithm is proposed to mitigate gradient backpropagation deviation in quantized components.
• Experimental models include Faster R-CNN, SSD, and Tiny YOLO v2 with diverse network architectures.

Main Results:

• Direct quantization resulted in a 2% mean average precision (mAP) drop with specific bit-width constraints.
• Retraining quantized models using IVS-Caffe reduced the mAP drop to less than 1% under stricter bit-width constraints.
• The tool accurately analyzes model accuracy on hardware accelerators with different bit widths.

Conclusions:

• IVS-Caffe effectively enables the training of bit-accurate CNN models, preserving accuracy at reduced bit widths.
• The tool facilitates fine-tuning CNN models or customizing hardware accelerators for lower power consumption.
• This work provides a valuable solution for efficient CNN deployment in hardware-constrained environments.