Mapping Neural Networks to FPGA-Based IoT Devices for Ultra-Low Latency Processing

Area of Science:

• Computer Engineering
• Artificial Intelligence
• Hardware Acceleration

Background:

• Internet of Things (IoT) infrastructure demands rapid knowledge access.
• Critical applications like robotics and autonomous driving prioritize system response time for Quality of Service (QoS).
• Existing solutions often struggle to meet stringent latency requirements for complex neural models.

Purpose of the Study:

• To propose a methodology and framework for mapping neural models to Field-Programmable Gate Arrays (FPGAs).
• To focus on minimizing latency for real-time IoT applications.
• To enable efficient hardware deployment of neural networks.

Main Methods:

• Utilized Multi-objective Covariance Matrix Adaptation Evolution Strategy (MO-CMA-ES).
• Employed custom scores for model sparsity, bit-width, and quality.
• Developed a framework for mapping neural models to FPGAs.
• Validated the solution on a Xilinx Zynq UltraScale+ MPSoC platform.

Main Results:

• Achieved a latency of 210 ns for a specific neural model (two LSTM, one dense layer).
• Demonstrated compression ratios through quantization and pruning, with and without retraining.
• Validated the framework's effectiveness across three case studies.
• Provided a publicly available framework for FPGA model mapping.

Conclusions:

• The proposed methodology effectively reduces latency in mapping neural models to FPGAs.
• The framework facilitates efficient deployment of AI models for real-time IoT applications.
• The achieved low latency is crucial for applications demanding high QoS.