Mixed-Precision Deep Learning Based on Computational Memory.

Area of Science:

• Artificial Intelligence
• Computer Architecture
• Materials Science

Background:

• Deep neural networks (DNNs) require intensive computation, driving the need for efficient hardware.
• Computational memory using resistive devices offers in-memory computing but faces challenges in reliable weight updates.
• Non-von Neumann architectures are explored to overcome the limitations of traditional computing for AI.

Purpose of the Study:

• To propose a mixed-precision architecture combining computational memory with digital processing for DNN training.
• To address the challenge of imprecise conductance updates in memory devices during DNN training.
• To enhance the energy efficiency and accuracy of AI hardware accelerators.

Main Methods:

• A mixed-precision architecture integrating a computational memory unit and a digital processing unit was designed.
• Phase-change memory (PCM) arrays were used for synaptic weight storage and in-memory summation.
• Hardware/software co-training experiments were conducted on multilayer perceptrons and other network types.

Main Results:

• The proposed architecture achieved 97.73% test accuracy on MNIST handwritten digit classification, closely matching software baselines.
• Comparable accuracies to floating-point implementations were achieved across various network types (CNNs, LSTMs, GANs).
• A 172x improvement in energy efficiency was demonstrated for training a multilayer perceptron compared to digital implementations.

Conclusions:

• The mixed-precision architecture effectively overcomes the limitations of imprecise memory updates in computational memory for DNN training.
• This approach offers a viable path towards highly accurate and energy-efficient AI hardware.
• The architecture demonstrates broad applicability across diverse deep learning models.