Accurate deep neural network inference using computational phase-change memory

Area of Science:

• Neuromorphic Engineering
• Computer Architecture
• Materials Science

Background:

• In-memory computing with resistive memory devices offers energy-efficient, non-von Neumann hardware for deep learning.
• Device variability and noise in analog memory pose challenges for accurate weight transfer from digital training, impacting neural network performance.
• Maintaining accuracy during the transition from digital training to analog hardware implementation is crucial for practical applications.

Purpose of the Study:

• To develop a training methodology for ResNet-type convolutional neural networks (CNNs) that ensures minimal accuracy loss when weights are mapped to phase-change memory (PCM) devices.
• To propose a compensation technique using batch normalization parameters to enhance accuracy retention over time in PCM-based systems.
• To demonstrate the feasibility and effectiveness of the proposed methods through benchmark datasets and hardware experiments.

Main Methods:

• A specialized training methodology was developed for ResNet-like CNNs, optimizing weights for transfer to analog PCM devices.
• A compensation technique leveraging batch normalization parameters was implemented to mitigate accuracy degradation over time.
• The methodology was validated by mapping trained weights to PCM devices and evaluating performance on CIFAR-10 and ImageNet datasets.

Main Results:

• Achieved 93.7% classification accuracy on CIFAR-10 and 71.6% top-1 accuracy on ImageNet after mapping trained weights to PCM.
• Hardware experiments with ResNet-32 on CIFAR-10 demonstrated sustained accuracy above 93.5% over a one-day period.
• Successfully programmed 361,722 synaptic weights using only two PCM devices per weight in a differential configuration.

Conclusions:

• The proposed training methodology effectively addresses the accuracy loss issue in transferring digital weights to analog PCM devices for in-memory computing.
• The batch normalization-based compensation technique improves long-term accuracy retention in neuromorphic hardware.
• This work paves the way for highly energy-efficient and accurate deep learning inference hardware utilizing resistive memory technologies.