High-performance deep spiking neural networks with 0.3 spikes per neuron.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Biological brains achieve energy efficiency through rare, binary spikes.
• Training biologically-inspired spiking neural networks (SNNs) is challenging compared to artificial neural networks (ANNs).
• Theoretical mappings exist from ANNs to SNNs using time-to-first-spike coding, but practical training remains difficult.

Purpose of the Study:

• Analyze the learning dynamics of time-to-first-spike SNNs.
• Identify and address the vanishing/exploding gradient problem in SNN training.
• Demonstrate effective training of deep SNNs for high-performance image classification.

Main Methods:

• Theoretical analysis of SNN learning dynamics.
• Simulation of SNNs with a focus on neuron membrane potential at threshold.
• Utilizing a robust gradient descent algorithm for training and fine-tuning.
• Testing on image datasets including MNIST, Fashion-MNIST, CIFAR10, CIFAR100, and PLACES365.

Main Results:

• Identified a specific vanishing/exploding gradient problem in SNNs.
• A constant membrane potential slope at threshold ensures training equivalence with ANNs (using rectified linear units).
• Deep SNNs achieved performance matching ANNs on various image classification benchmarks.
• High-performance classification was achieved using fewer than 0.3 spikes per neuron.

Conclusions:

• Deep SNNs can be effectively trained from scratch or fine-tuned to match ANN performance.
• The proposed method enables energy-efficient AI implementations.
• Optimized SNNs show potential for low-latency, noise-resilient hardware applications.