Efficient Neural Networks on the Edge with FPGAs by Optimizing an Adaptive Activation Function.

Area of Science:

• Edge computing
• Embedded systems
• Machine learning hardware acceleration

Background:

• Deep neural networks (DNNs) on edge devices face computational and memory challenges.
• Shallow neural networks offer efficiency but often lack accuracy compared to DNNs.
• Adaptive activation functions (AAFs) are crucial for enhancing shallow network performance.

Purpose of the Study:

• To demonstrate that a customized adaptive activation function (AAF) can enable shallow neural networks to achieve DNN-level accuracy.
• To design an efficient FPGA implementation of a segmented spline curve neural network (SSCNN) utilizing an AAF.
• To validate the performance of the proposed SSCNN for real-time digital predistortion in RF power amplifiers.

Main Methods:

• Developed a customized segmented spline curve neural network (SSCNN) structure with an adaptive activation function (AAF).
• Implemented the SSCNN on an FPGA, comparing it against real-valued time-delay neural networks (RVTDNNs), augmented RVTDNNs (ARVTDNNs), and DNNs.
• Utilized the AMD/Xilinx RFSoC ZCU111 for experimental validation in radio-frequency (RF) digital predistortion applications.

Main Results:

• The SSCNN implementation achieved similar accuracy to DNNs while using 40% fewer hardware resources and no block RAMs.
• The FPGA implementation consumed less than 3% of available resources on the RFSoC ZCU111.
• The solution enabled a clock frequency increase to 221.12 MHz, facilitating wide bandwidth signal transmission.

Conclusions:

• Customized shallow neural networks with AAFs provide a computationally efficient and memory-saving alternative to DNNs for edge devices.
• The proposed SSCNN offers a viable hardware acceleration solution for real-time RF signal processing tasks like digital predistortion.
• The FPGA implementation demonstrates the potential for deploying accurate and efficient AI models on resource-constrained edge platforms.