AoCStream: All-on-Chip CNN Accelerator with Stream-Based Line-Buffer Architecture and Accelerator-Aware Pruning

Area of Science:

• Computer Science
• Artificial Intelligence
• Embedded Systems

Background:

• Convolutional Neural Networks (CNNs) are vital for EdgeAI and TinyML but demand significant memory.
• External memory requirements hinder the feasibility of CNNs in resource-limited EdgeAI and TinyML applications.

Purpose of the Study:

• To propose novel memory-reduction methods for CNNs at both algorithmic and architectural levels.
• To enable the implementation of performant CNNs using only the on-chip memory of practical devices.

Main Methods:

• Implemented accelerator-aware pruning to reduce weight memory.
• Proposed a stream-based line-buffer architecture for activation memory reduction.
• Utilized a pipelined approach with dedicated layer blocks and line buffers instead of frame buffers.

Main Results:

• Successfully reduced intermediate data storage size.
• Demonstrated the implementation of object-detection CNNs (MobileNetV1/V2, SSDLite variant) on a low-end FPGA.
• Achieved implementation without requiring external memory.

Conclusions:

• The proposed memory-reduction techniques are effective for deploying CNNs in resource-constrained EdgeAI and TinyML environments.
• On-chip memory implementation of CNNs is feasible for practical applications like object detection on low-end FPGAs.