ETA: An Efficient Training Accelerator for DNNs Based on Hardware-Algorithm Co-Optimization.

Area of Science:

• Computer Engineering
• Artificial Intelligence
• Hardware Acceleration

Background:

• Deep neural network (DNN) training demands substantial computational resources and data access, posing challenges for resource-constrained platforms and user privacy.
• Efficient training on edge devices is crucial for real-time applications and data security.

Purpose of the Study:

• To develop an efficient training accelerator (ETA) on field-programmable gate arrays (FPGAs) for resource-constrained platforms.
• To enable efficient DNN training with reduced computational complexity and memory access through hardware-algorithm co-optimization.

Main Methods:

• Implemented an efficient training accelerator (ETA) on FPGA using a hardware-algorithm co-optimization approach.
• Introduced a novel 8-bit precision training scheme with a compact data format and hardware-oriented normalization layer.
• Designed a reconfigurable processing element (PE), flexible network-on-chip (NoC), hierarchical PE array, and a unified computing core for auxiliary layers.

Main Results:

• Achieved state-of-the-art accuracy across multiple DNN models (CIFAR-VGG16, CIFAR-ResNet20, CIFAR-InceptionV3, ResNet18, ResNet50).
• Demonstrated high throughput on Xilinx VC709 FPGA: 610.98 GOPS (CIFAR-VGG16), 658.64 GOPS (CIFAR-ResNet20), 811.24 GOPS (ResNet18).
• Achieved a 3.65× speedup and 8.54× energy efficiency improvement compared to prior art on CIFAR-ResNet20.

Conclusions:

• The proposed hardware-algorithm co-optimization approach and novel training scheme significantly enhance DNN training efficiency on FPGAs.
• The ETA effectively reduces computational complexity and memory access, enabling high performance and energy efficiency for DNNs on resource-constrained platforms.