Noise-Tolerant CIM-DNNs Explained.

Area of Science:

• Computer Engineering
• Artificial Intelligence
• Materials Science

Background:

• Compute-in-memory (CIM) systems using resistive random access memory (RRAM) accelerate deep neural network (DNN) computations.
• RRAM-based CIM systems face computational errors due to analog computing's inherent noise and device variations.
• Existing noise mitigation techniques, particularly variation-aware training (VAT), show promise but lack theoretical understanding for optimal implementation.

Purpose of the Study:

• To investigate the fundamental properties of noise tolerance in DNNs for CIM systems.
• To identify key factors influencing DNN performance under noise and elucidate the mechanisms of VAT.
• To provide practical guidelines for implementing VAT to achieve optimal noise tolerance in CIM-DNNs.

Main Methods:

• Conducted series of training experiments to identify noise tolerance factors in DNNs.
• Developed theoretical insights and practical demonstrations of VAT's noise resistance mechanisms.
• Analyzed the impact of noise injection and optimizers on convergence dynamics during VAT.

Main Results:

• Identified critical factors influencing DNN noise tolerance through empirical training.
• Demonstrated theoretically and practically how VAT enhances resistance to noise during training.
• Established foundational understanding of VAT for improving noise tolerance in CIM-DNNs.

Conclusions:

• VAT is a promising, low-cost method for creating noise-tolerant CIM-DNNs.
• Understanding noise tolerance properties is crucial for advancing VAT.
• This work provides guidelines for optimal VAT implementation, contributing to reliable CIM systems.