Analog Resistive Switching Devices for Training Deep Neural Networks with the Novel Tiki-Taka Algorithm.

Area of Science:

• Materials Science
• Computer Engineering
• Artificial Intelligence

Background:

• Training large neural networks (NNs) is hindered by communication bottlenecks with off-chip memory.
• Integrating analog memory crossbar arrays in the Back-End-Of-Line (BEOL) offers a solution for storing NN parameters and performing synaptic operations.
• The Tiki-Taka algorithm aids NN training despite device imperfections, but requires specific resistive switching device characteristics.

Purpose of the Study:

• To develop and demonstrate a novel resistive switching device compatible with the Tiki-Taka algorithm for enhanced NN training.
• To address the limitations of existing devices that fail to meet Tiki-Taka requirements: multiple programmable states, a centered symmetry point, and low programming noise.

Main Methods:

• Fabrication of a complementary metal-oxide semiconductor (CMOS)-compatible resistive random-access memory (RRAM) device.
• Characterization of the RRAM device for programmable states, programming noise, and symmetry point accuracy.
• Evaluation of the device's suitability for implementing the Tiki-Taka algorithm.

Main Results:

• The developed RRAM exhibits over 30 programmable states with low programming noise.
• The device demonstrates a symmetry point with only a 5% skew from the center.
• This is the first demonstration of a resistive switching device meeting all fundamental Tiki-Taka requirements.

Conclusions:

• The novel CMOS-compatible RRAM device successfully meets the Tiki-Taka algorithm's stringent requirements.
• These findings enable the generalization of Tiki-Taka training from small fully connected networks to larger neural network architectures, such as long/short-term memory (LSTM) networks.