A Low-Power General Matrix Multiplication Accelerator with Sparse Weight-and-Output Stationary Dataflow

Area of Science:

• Computer Science
• Machine Learning Hardware

Background:

• General Matrix Multiplication (GEMM) is computationally intensive, limiting its use in resource-constrained environments.
• Existing data reuse methods for GEMM do not fully exploit potential reductions in data movement.

Purpose of the Study:

• To develop a sparse GEMM accelerator that improves efficiency for machine learning on edge devices.
• To address challenges in storing and processing sparse matrices in hardware.

Main Methods:

• Introduced a weight-and-output stationary (WOS) dataflow and distributed buffer architecture for sparse GEMM.
• Developed an adaptable mapping scheme for compressed GEMM and an offline sparsity-aware shuffle strategy for weights.
• Implemented a low-cost sparse computing method with globally shared inputs for high throughput.

Main Results:

• The sparse GEMM accelerator processes matrices in a compressed format, eliminating on-chip weight and partial sum transfers.
• The sparsity-aware shuffle strategy balances buffer utilization and minimizes waste for irregular weight matrices.
• FPGA experiments demonstrated 1.73x better computing efficiency and 1.36x higher energy efficiency compared to existing methods.

Conclusions:

• The proposed sparse GEMM accelerator effectively reduces data movement and improves hardware efficiency for machine learning.
• The novel dataflow, buffer architecture, and weight management strategy enable efficient processing of sparse matrices.
• This work offers a viable solution for deploying computationally demanding machine learning models on resource-limited platforms.