Trainable quantization for Speedy Spiking Neural Networks.

Area of Science:

• Neuromorphic Engineering
• Artificial Intelligence
• Computer Science

Background:

• Spiking Neural Networks (SNNs) are the third generation of Artificial Neural Networks, utilizing binary, asynchronous spikes for computation, offering theoretical sparse and low-power operations.
• Despite research interest, SNNs face limitations including inefficient learning rules, immature frameworks, and significant data processing latency, which generates energy overhead.
• While learning rules and frameworks have improved, the core issue of latency, stemming from gradual spike propagation, remains a major challenge, impacting overall efficiency.

Purpose of the Study:

• To address the persistent latency problem in Spiking Neural Networks by focusing on quantization error.
• To propose an in-depth characterization of SNN quantization noise and its impact on accuracy.
• To introduce a novel end-to-end direct learning approach with a trainable spiking neural model to minimize quantization noise and reduce latency.

Main Methods:

• Characterization of quantization error and noise in Spiking Neural Networks.
• Development of a novel trainable spiking neural model capable of adapting neuron thresholds during training.
• Implementation of efficient quantization strategies within the new model for end-to-end learning.

Main Results:

• The proposed model effectively characterizes and minimizes quantization noise, identified as a primary cause of accuracy drop in SNNs compared to Artificial Neural Networks (ANNs).
• The novel approach leads to Spiking Neural Networks that achieve state-of-the-art accuracy on benchmark datasets.
• The resulting SNNs demonstrate reduced latency by training over a limited number of timesteps while preserving high sparsity.

Conclusions:

• Quantization error is a critical factor limiting Spiking Neural Network accuracy and performance.
• The developed trainable spiking neural model and learning approach effectively mitigate quantization noise, leading to improved accuracy and reduced latency.
• This work advances neuromorphic computing by enabling efficient, accurate, and low-latency Spiking Neural Networks suitable for real-world applications.