Bulk-Switching Memristor-Based Compute-In-Memory Module for Deep Neural Network Training.

Area of Science:

• Electrical Engineering
• Computer Science
• Materials Science

Background:

• Deep neural networks (DNNs) require significant computational resources for training.
• Memristor-based compute-in-memory (CIM) offers potential for efficient DNN inference but faces challenges in training due to device non-linearities and precision limitations.

Purpose of the Study:

• To experimentally implement and validate a mixed-precision training scheme for DNNs using memristor-based CIM modules.
• To address challenges in CIM-based DNN training, including non-linear weight updates, device variations, and low precision.

Main Methods:

• A mixed-precision training scheme was developed, utilizing low-precision CIM modules for accelerated vector-matrix multiplication (VMM) and high-precision digital units for accumulating weight updates.
• Memristor devices were updated only when accumulated weight changes exceeded a threshold.
• The scheme was implemented on a system-on-chip integrating analog CIM modules and digital sub-systems.

Main Results:

• The proposed scheme demonstrated fast convergence for LeNet training, achieving 97.73% accuracy.
• Evaluations with realistic hardware parameters confirmed the efficacy of CIM modules for efficient mixed-precision DNN training.
• Models trained on-chip showed robustness to hardware variations, enabling direct deployment on CIM inference chips.

Conclusions:

• Mixed-precision training using memristor-based CIM modules enables efficient and accurate DNN training.
• This approach mitigates hardware variations and simplifies the deployment of trained models on CIM inference hardware.
• The developed system-on-chip facilitates practical implementation of advanced AI models.