Acceleration of Deep Neural Network Training Using Field Programmable Gate Arrays

Area of Science:

• Computer Engineering
• Artificial Intelligence
• Hardware Acceleration

Background:

• Convolutional Neural Network (CNN) training demands significant computational resources.
• Field-Programmable Gate Arrays (FPGAs) show promise for efficient CNN inference and training.
• Scalability challenges exist due to hardware limitations for large datasets and models.

Purpose of the Study:

• To propose an energy-efficient procedure for CNN training utilizing FPGA-based accelerators.
• To investigate the impact of optimizations like quantization and gradient accumulation on training efficiency.
• To compare the performance and energy efficiency of FPGA accelerators against GPUs and CPUs.

Main Methods:

• Implemented a CNN training procedure on an FPGA accelerator.
• Employed quantization as a model compression technique to enhance training speed.
• Utilized a gradient accumulation buffer for optimal operating efficiency during gradient descent.
• Validated the design using AlexNet and VGG-16 models on an FPGA board, laptop CPU, and GPU.

Main Results:

• Achieved 203.75 GOPS with AlexNet and 196.50 GOPS with VGG-16 on the Terasic DE1 SoC FPGA.
• Demonstrated superior energy efficiency of FPGA accelerators compared to other tested platforms.
• Quantization and gradient accumulation effectively sped up the CNN training process.

Conclusions:

• FPGA-based accelerators offer a viable solution for energy-efficient CNN training.
• The proposed optimizations enhance computational performance and reduce power consumption.
• FPGA accelerators provide a scalable hardware solution for demanding deep learning workloads.