Neural Network Training With Asymmetric Crosspoint Elements.

Area of Science:

• Materials Science
• Computer Science
• Artificial Intelligence

Background:

• Analog crossbar arrays with programmable non-volatile resistors are key for accelerating deep neural network (DNN) training.
• Asymmetric conductance modulation in practical resistive devices significantly impairs DNN classification performance with conventional training algorithms.

Purpose of the Study:

• To identify the fundamental reasons for incompatibility between asymmetric resistive devices and conventional DNN training.
• To present a novel, fully-parallel training algorithm compatible with asymmetric crosspoint elements for analog deep learning.

Main Methods:

• Detailed analysis of the fundamental incompatibilities between device asymmetry and conventional training methods.
• Development of a novel fully-parallel training algorithm based on principles analogous to classical mechanics.
• Exploitation of device asymmetry as a feature within the new algorithm, minimizing system energy (Hamiltonian) instead of solely relying on error function gradients.

Main Results:

• The study elucidates the core reasons for performance degradation caused by asymmetric conductance modulation.
• A new training algorithm is theoretically established, designed to be compatible with asymmetric crosspoint elements.
• The proposed method demonstrates how device asymmetry can be leveraged as a beneficial feature in analog deep learning processors.

Conclusions:

• The developed algorithm effectively addresses the limitations imposed by asymmetric resistive devices in analog DNN accelerators.
• This technique facilitates the immediate implementation of analog deep learning accelerators using existing device technologies.
• The findings pave the way for more efficient and robust analog computing hardware for AI applications.