Surrogate gradients for analog neuromorphic computing.

Area of Science:

• Neuromorphic Engineering
• Computational Neuroscience
• Artificial Intelligence

Background:

• Biological neurons process information via analog summation and binary spikes, inspiring energy-efficient spiking neural networks (SNNs).
• Analog neuromorphic hardware emulates SNNs but faces challenges with device variability and training algorithm integration.
• Surrogate gradient learning (SGL) is a key training method for SNNs, yet its application to analog systems was unproven.

Purpose of the Study:

• To demonstrate the efficacy of surrogate gradient learning for training SNNs on the BrainScaleS-2 analog neuromorphic system.
• To investigate if SGL can overcome device mismatch inherent in analog hardware.
• To establish performance benchmarks for low-power, high-speed SNNs on analog platforms.

Main Methods:

• Implemented an in-the-loop surrogate gradient learning approach on the BrainScaleS-2 analog neuromorphic system.
• Trained SNNs for vision and speech processing tasks, evaluating performance and energy efficiency.
• Analyzed the network's ability to self-correct for device mismatch during training.

Main Results:

• Successfully demonstrated SGL on analog neuromorphic hardware, achieving competitive performance on vision and speech benchmarks.
• Showcased that SGL effectively self-corrects for device mismatch, a critical challenge in analog systems.
• Achieved highly sparse spiking activity ( < 1 spike/neuron/input), high inference rates (up to 85k frames/sec), and low power consumption (< 200 mW).

Conclusions:

• Surrogate gradient learning is a viable and effective training strategy for analog neuromorphic systems.
• This work establishes new benchmarks for energy-efficient and high-performance spiking network processing on analog hardware.
• The findings pave the way for developing advanced on-chip learning algorithms for future neuromorphic systems.