SALMON: Self-Adaptive Learning Model on Neuromorphic Hardware

Area of Science:

• Neuromorphic Engineering
• Artificial Intelligence Hardware
• Computer Architecture

Background:

• Analog in-memory computing (AIMC) offers energy-efficient deep learning acceleration, primarily for inference.
• On-chip training for AIMC faces challenges due to inherent hardware nonidealities.
• Existing methods lack robust solutions for training complex models directly on neuromorphic systems.

Purpose of the Study:

• Introduce a novel on-chip training method, Self-adaptive learning model on neuromorphic hardware (SALMON), for large-scale neuromorphic systems.
• Address hardware nonidealities in AIMC to enable effective on-chip training.
• Enhance the performance and efficiency of deep learning training on neuromorphic hardware.

Main Methods:

• Developed SALMON, a self-adaptive network (SAnet) integrating an analog backbone with digital attention blocks.
• Employed ResNet architecture for CIFAR-10 image classification with varying analog device nonideality levels.
• Utilized Grad-CAM analysis for ablation studies on digital attention block contributions.

Main Results:

• Achieved test accuracy up to 91.49% in CIFAR-10 image classification.
• Demonstrated a 13.1% performance improvement across different network scales and nonideality levels.
• Showcased approximately 70% reduction in relative power consumption through optimized use of digital components.

Conclusions:

• SALMON effectively enables on-chip training for AIMC by mitigating hardware nonidealities.
• The integration of digital attention blocks significantly enhances network performance.
• Optimized application strategies for digital components drastically reduce power consumption during on-chip training.