Accurate deep neural network inference using computational phase-change memory | JoVE Visualize

Area of Science:

Neuromorphic Engineering
Computer Architecture
Materials Science

Background:

In-memory computing with resistive memory devices offers energy-efficient, non-von Neumann hardware for deep learning.
Device variability and noise in analog memory pose challenges for accurate weight transfer from digital training, impacting neural network performance.
Maintaining accuracy during the transition from digital training to analog hardware implementation is crucial for practical applications.

Purpose of the Study:

To develop a training methodology for ResNet-type convolutional neural networks (CNNs) that ensures minimal accuracy loss when weights are mapped to phase-change memory (PCM) devices.
To propose a compensation technique using batch normalization parameters to enhance accuracy retention over time in PCM-based systems.
To demonstrate the feasibility and effectiveness of the proposed methods through benchmark datasets and hardware experiments.

Main Methods:

A specialized training methodology was developed for ResNet-like CNNs, optimizing weights for transfer to analog PCM devices.
A compensation technique leveraging batch normalization parameters was implemented to mitigate accuracy degradation over time.
The methodology was validated by mapping trained weights to PCM devices and evaluating performance on CIFAR-10 and ImageNet datasets.

Main Results:

Achieved 93.7% classification accuracy on CIFAR-10 and 71.6% top-1 accuracy on ImageNet after mapping trained weights to PCM.
Hardware experiments with ResNet-32 on CIFAR-10 demonstrated sustained accuracy above 93.5% over a one-day period.
Successfully programmed 361,722 synaptic weights using only two PCM devices per weight in a differential configuration.

Conclusions:

The proposed training methodology effectively addresses the accuracy loss issue in transferring digital weights to analog PCM devices for in-memory computing.
The batch normalization-based compensation technique improves long-term accuracy retention in neuromorphic hardware.
This work paves the way for highly energy-efficient and accurate deep learning inference hardware utilizing resistive memory technologies.