Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based...

Area of Science:

• Hardware-aware machine learning
• Energy-efficient computing architectures
• Deep learning acceleration

Background:

• Analog in-memory computing (AIMC) offers energy-efficient deep learning acceleration by performing matrix-vector multiplications.
• Non-ideal device characteristics in AIMC can lead to non-deterministic or non-linear operations, potentially reducing inference accuracy.
• Existing retraining methods may not fully account for the complex non-idealities present in analog hardware.

Purpose of the Study:

• To develop a hardware-aware retraining approach for systematically evaluating analog in-memory computing accuracy.
• To investigate the sensitivity and robustness of deep neural networks to various non-idealities in AIMC.
• To demonstrate that deep neural networks can achieve comparable accuracy to floating-point implementations using AIMC.

Main Methods:

• A hardware-aware retraining methodology was developed to optimize deep neural networks for AIMC.
• A realistic crossbar model was integrated to simulate AIMC non-idealities.
• Multiple network topologies, including convolutional neural networks (convnets), recurrent neural networks (RNNs), and transformers, were analyzed.
• Sensitivity analysis was performed across a range of non-ideal conditions.

Main Results:

• Many large-scale deep neural networks, including convnets, RNNs, and transformers, can be successfully retrained to achieve iso-accuracy compared to floating-point implementations.
• Non-idealities affecting inputs or outputs have a greater impact on accuracy than those affecting weights.
• Recurrent neural networks demonstrated particular robustness against all investigated non-idealities.

Conclusions:

• Hardware-aware retraining is effective in mitigating accuracy degradation in analog in-memory computing for deep learning.
• Input/output noise significantly impacts AIMC accuracy, highlighting areas for hardware and algorithm co-design.
• Recurrent networks are a promising architecture for robust and accurate analog in-memory computing applications.