HCG: Streaming DCNN Accelerator With a Hybrid Computational Granularity Scheme on FPGA

Area of Science:

• Computer Engineering
• Hardware Acceleration
• Deep Learning

Background:

• Streaming Deep Convolutional Neural Network (DCNN) accelerators on FPGAs utilize interconvolutional-layer parallelism to boost throughput.
• Existing accelerators often use uniform tiling methods, impacting the memory-bandwidth tradeoff and FPGA resource utilization.
• Computational granularity, the size of output feature maps computed per tiled input feature map, is a key factor influencing performance.

Purpose of the Study:

• To introduce a novel Hybrid Computational Granularity (HCG) scheme for streaming DCNN accelerators.
• To analyze the impact of HCG on system pipelines, aiming to eliminate pipeline bubbles and simplify design.
• To develop a hardware overhead model and heuristic algorithm for optimizing HCG and achieving better memory-bandwidth tradeoffs and throughput.

Main Methods:

• Developed a Hybrid Computational Granularity (HCG) scheme for independently optimized computational granularity per node.
• Theoretically analyzed the impact of computational granularity on system pipelines and control logic.
• Created a hardware overhead model and heuristic algorithm for optimizing granularity.
• Implemented and validated accelerators on an Alveo U250 board under various bandwidth constraints.

Main Results:

• The HCG scheme enables a more flexible memory-bandwidth tradeoff and effective FPGA resource utilization.
• The proposed methodology successfully established seamless system pipelines without bubbles, simplifying design.
• Implemented accelerators achieved high throughputs (e.g., 3-TOPS ResNet-18) surpassing existing works under specific bandwidth constraints.
• Demonstrated accelerators for various DCNN models including VGG-16, ResNet-34/50, MobileNetV1, ConvNeXt-T, and ResNeXt-50.

Conclusions:

• The HCG scheme is an effective approach for optimizing streaming DCNN accelerators on FPGAs.
• The developed optimization methodology achieves superior memory-bandwidth tradeoffs and higher throughput.
• The proposed design and validation methodology demonstrate significant advancements in FPGA-based deep learning acceleration.