A methodology for accurate benchmarking of neural network accelerators using a high-level synthesis-based hardware...

Area of Science:

• Computer Engineering
• Artificial Intelligence Hardware

Background:

• Neural network models are increasing in complexity, driving demand for specialized hardware accelerators.
• Existing evaluation metrics (e.g., TOPS, TOPS/W) are insufficient for fair comparisons due to external factors like technology node and workload variations.

Purpose of the Study:

• To propose a standardized methodology for benchmarking neural network accelerators.
• To enable fair, apples-to-apples comparisons across diverse accelerator architectures.

Main Methods:

• Developed Voyager, a high-level synthesis (HLS)-based accelerator generator.
• Created baseline accelerators with matched compute scale and technology nodes.
• Ran identical workloads on generated accelerators for comparative analysis.

Main Results:

• Voyager enables the creation of well-optimized baseline accelerators.
• The methodology facilitates systematic evaluation of accelerator architectures.
• Case studies demonstrated effective comparisons across different accelerator types, including in-memory and sparsity-aware designs.

Conclusions:

• The proposed benchmarking methodology using Voyager provides a robust framework for evaluating neural network accelerators.
• This approach allows for isolation of architectural impacts, leading to more meaningful analysis and design insights.