PULP-NN: accelerating quantized neural networks on parallel ultra-low-power RISC-V processors

Area of Science:

• Computer Engineering
• Artificial Intelligence
• Embedded Systems

Background:

• Deep neural network (DNN) inference demands efficient computation, especially on resource-constrained edge devices.
• Aggressive quantization (using fewer bits for weights and activations) is a key trend to reduce DNN computational and memory footprints.
• RISC-V processors, with their open-source nature and customizability, are increasingly adopted for ultra-low-power embedded applications.

Purpose of the Study:

• To introduce PULP-NN, an optimized computing library for parallel, ultra-low-power RISC-V processor clusters.
• To enable efficient inference of highly quantized neural networks (down to INT-1) on these platforms.
• To leverage RISC-V digital signal processing extensions and cluster parallelism for maximum performance.

Main Methods:

• Development of specialized kernels for quantized neural network inference targeting byte and sub-byte data types.
• Exploitation of parallel processing capabilities in tightly coupled RISC-V clusters.
• Optimization using digital signal processing (DSP) extensions available on PULP RISC-V processors.

Main Results:

• PULP-NN achieves up to 15.5 MACs/cycle on INT-8 data types.
• Performance improvements of up to 63x compared to sequential implementations on a single RISC-V core.
• Significant speedups (30x and 19.6x) over ARM CMSIS-NN on STM32L4/H7 for CIFAR-10 networks.
• Outperforms STM32L4 and STM32H7 by 36.8x and 7.45x respectively on GAP-8 at maximum frequency.
• Achieves 14.1x and 39.5x higher energy efficiency on GAP-8 compared to STM32L4 and STM32H7.

Conclusions:

• PULP-NN offers a highly optimized solution for efficient neural network inference on parallel RISC-V clusters.
• The library effectively supports aggressive quantization, enabling significant performance and energy savings.
• PULP-NN represents a substantial advancement for energy-autonomous computing and intelligence in edge devices.