Equivalent-accuracy accelerated neural-network training using analogue memory.

Area of Science:

• Materials Science
• Computer Science
• Electrical Engineering

Background:

• Neural network training is energy-intensive due to data transfer between memory and processors.
• Analog non-volatile memory offers potential for in situ training but faces accuracy limitations.
• Existing analog memory hardware shows insufficient dynamic range and weight-update asymmetry.

Purpose of the Study:

• To develop a mixed hardware-software system for efficient neural network training.
• To overcome accuracy limitations of in situ training using non-volatile memory.
• To enhance computational energy efficiency and throughput for neural network accelerators.

Main Methods:

• Implemented mixed hardware-software neural networks with up to 204,900 synapses.
• Utilized phase-change memory for long-term storage and volatile capacitors for near-linear updates.
• Employed 'polarity inversion' for weight-data transfer to mitigate device variations.

Main Results:

• Achieved generalization accuracies equivalent to software-based training on standard machine learning datasets (MNIST, CIFAR-10/100).
• Demonstrated computational energy efficiency of 28,065 billion operations/sec/watt.
• Calculated throughput per area exceeding current graphical processing units by two orders of magnitude.

Conclusions:

• The developed system provides a viable path towards fast and energy-efficient hardware accelerators for neural networks.
• This approach is particularly effective for fully connected neural network layers.
• The findings pave the way for next-generation AI hardware.