Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices.

Area of Science:

• Neuromorphic engineering
• Artificial intelligence
• Computer science

Background:

• Deep neural networks (DNNs) offer significant performance benefits when implemented using resistive device arrays.
• Resistive Processing Units (RPUs) have been previously detailed for fully connected DNNs.

Purpose of the Study:

• To extend the RPU concept to convolutional neural networks (CNNs).
• To demonstrate the mapping of convolutional layers to RPU arrays for efficient hardware parallelism.
• To address and mitigate the impact of analog computation limitations on CNN training accuracy.

Main Methods:

• Mapping convolutional layers to fully connected RPU arrays.
• Implementing noise and bound management techniques for analog computations.
• Utilizing digitally programmable update management and device variability reduction.

Main Results:

• Successful utilization of hardware parallelism across all backpropagation cycles.
• Identification of significant impact of analog noise and bound limitations on CNN training accuracy.
• Demonstration of noise and bound management techniques mitigating accuracy degradation without analog circuit complexity.
• Validation of RPU concept for training CNNs through a combination of proposed techniques.

Conclusions:

• The RPU concept can be successfully applied to train CNNs by employing specific noise and bound management strategies.
• The presented techniques are general and applicable beyond CNNs, broadening the scope of RPU applications.
• This work enables the RPU approach for a wide range of neural network architectures, enhancing hardware-accelerated deep learning.