DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Deep Learning

Background:

• Spiking neural networks (SNNs) offer computational efficiency on event-driven hardware due to their asynchronous, event-based processing.
• Current SNNs face challenges with high inference latency, stemming from inefficient input encoding and suboptimal neuron parameter settings (firing threshold, membrane leak).

Purpose of the Study:

• To introduce DIET-SNN, a deep spiking network designed to minimize inference latency and enhance computational efficiency.
• To optimize neuron parameters, specifically membrane leak and firing threshold, alongside network weights using gradient descent.

Main Methods:

• DIET-SNN employs end-to-end backpropagation to optimize membrane leak and firing threshold for each layer.
• The input layer directly processes analog pixel values, with the first convolutional layer converting these to spikes using leaky-integrate-and-fire (LIF) neurons.
• Trained membrane leak selectively attenuates membrane potential, increasing activation sparsity.

Main Results:

• DIET-SNN achieved 69% top-1 accuracy on ImageNet with only five timesteps (inference latency).
• The model demonstrated a 12x reduction in compute energy compared to standard artificial neural networks (ANNs).
• Inference speed was 20-500x faster than other state-of-the-art SNN models.

Conclusions:

• DIET-SNN effectively reduces latency and improves computational efficiency in spiking neural networks through optimized neuron parameters and direct analog input processing.
• The proposed method achieves competitive accuracy while significantly enhancing performance metrics, making SNNs more practical for real-world applications.