A Hybrid Scale-Up and Scale-Out Approach for Performance and Energy Efficiency Optimization in Systolic Array...

Area of Science:

• Computer Engineering
• Artificial Intelligence Hardware Acceleration

Background:

• Deep neural networks (DNNs) demand significant computational resources, challenging existing hardware accelerators.
• Systolic array accelerators, crucial for tensor operations in DNNs, traditionally use scale-up (larger arrays) or scale-out (parallel arrays) approaches.
• Neither scale-up nor scale-out alone can achieve optimal performance and energy efficiency across diverse DNN tasks.

Purpose of the Study:

• To propose a novel hybrid systolic array accelerator architecture.
• To address the limitations of existing scale-up and scale-out methods for DNN acceleration.
• To optimize both performance and energy efficiency for a wide range of DNN workloads.

Main Methods:

• Developed a hybrid approach integrating scale-up and scale-out techniques for systolic array accelerators.
• Utilized mapping space exploration within a multi-tenant environment to assign DNN operations.
• Configured specific systolic array modules for different DNN computational requirements.

Main Results:

• The proposed hybrid systolic array accelerator demonstrated significant improvements over TPUv3.
• Achieved an average reduction in energy consumption of up to 8%.
• Showcased an average improvement in throughput of up to 57% across various DNN models.

Conclusions:

• The hybrid scale-up and scale-out approach effectively balances performance and energy efficiency for DNN acceleration.
• This integrated architecture provides a more versatile and efficient solution for modern AI hardware demands.
• The findings suggest a promising direction for future development of specialized AI accelerators.