Efficient event-based delay learning in spiking neural networks

Area of Science:

• Artificial Intelligence
• Computational Neuroscience

Background:

• Spiking Neural Networks (SNNs) offer energy-efficient computation via sparse communication, contrasting with traditional Artificial Neural Networks (ANNs).
• SNNs are inherently recurrent due to stateful neurons, making them suitable for spatio-temporal processing, but their intrinsic memory is limited by time constants.
• Delays offer a powerful mechanism to extend memory in SNNs.

Purpose of the Study:

• To propose an event-based training method for SNNs incorporating delays, enabling precise gradient calculation for weights and delays.
• To introduce a novel delay learning algorithm applicable to recurrent SNNs.
• To demonstrate improved performance and efficiency of SNNs with learned delays.

Main Methods:

• Developed an event-based training method based on the EventProp formalism for SNNs with delays.
• Implemented a delay learning algorithm supporting multiple spikes per neuron and recurrent connections.
• Evaluated the method on sequence detection, Yin-Yang, Spiking Heidelberg Digits, Spiking Speech Commands, and Braille letter reading datasets.

Main Results:

• The proposed algorithm successfully optimized delays from suboptimal initial states.
• Classification accuracy was enhanced compared to SNNs without delays, particularly in smaller networks.
• The method demonstrated significant efficiency gains, using less than half the memory and being up to 26x faster than state-of-the-art delay-learning techniques.

Conclusions:

• Event-based training with learned delays is an effective method for improving SNN performance and efficiency.
• Recurrent delays are particularly advantageous for smaller SNN architectures.
• This approach offers a computationally efficient and memory-sparing alternative for training SNNs with delays.