Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI

Area of Science:

• Deep Learning and Artificial Intelligence
• Hardware Acceleration for Machine Learning
• Embedded Systems and Real-Time Computing

Background:

• Deep learning, particularly Convolutional Neural Networks (CNNs), offers significant advantages in various applications.
• Consumer Personal Computer (PC) hardware is often unsuitable for industrial environments due to harsh conditions and strict timing requirements.
• Field Programmable Gate Arrays (FPGAs) are gaining traction for efficient network inference in industrial settings.

Purpose of the Study:

• To propose a novel family of custom network architectures for real-time inference on FPGAs.
• To develop a trainable quantization layer (Requantizer) for integer arithmetic with customizable precision (down to two bits).
• To enable efficient training on Graphics Processing Units (GPUs) and subsequent synthesis to FPGA hardware.

Main Methods:

• Designed custom layers utilizing integer arithmetic with adjustable bit precision.
• Developed a trainable quantization layer, the 'Requantizer,' for non-linear activation and value rescaling.
• Trained models using TensorFlow Lite on GPUs and synthesized them for Xilinx FPGAs using Vivado.

Main Results:

• Achieved quantized network accuracy comparable to floating-point versions without requiring calibration data.
• Demonstrated superior performance compared to dedicated peak detection algorithms in a case study.
• FPGA implementation achieved real-time processing at four gigapixels per second with 0.5 TOPS/W efficiency.

Conclusions:

• The proposed custom FPGA solutions offer a viable alternative for real-time deep learning inference in industrial applications.
• The quantization-aware training approach effectively handles limited precision constraints.
• The developed hardware accelerators provide high performance and energy efficiency, competitive with custom integrated solutions.