Three factor delay learning rules for spiking neural networks.

Area of Science:

• Computational Neuroscience
• Machine Learning

Background:

• Spiking neural networks (SNNs) process spatiotemporal data but often lack effective temporal pattern recognition due to limited learnable parameters.
• Existing methods for incorporating delays require large networks and offline learning, hindering real-time applications.

Purpose of the Study:

• To introduce learnable synaptic and axonal delays into leaky integrate and fire (LIF)-based SNNs.
• To develop online three-factor learning rules for simultaneously learning weights and delays.
• To enable efficient, real-time temporal pattern recognition in resource-constrained environments.

Main Methods:

• Incorporated synaptic and axonal delays into feedforward and recurrent LIF SNNs.
• Proposed novel three-factor learning rules for online weight and delay adaptation.
• Utilized a smooth Gaussian surrogate for approximating spike derivatives in eligibility trace calculations.

Main Results:

• Delay incorporation improved accuracy by up to 18% compared to weights-only SNNs.
• Joint learning of weights and delays yielded up to 14% higher accuracy for comparable network sizes.
• Achieved competitive accuracy on the SHD speech recognition dataset, comparable to offline methods.
• Reduced model size by 6.6x and inference latency by 50% compared to state-of-the-art methods.

Conclusions:

• Jointly learning weights and delays in SNNs significantly enhances temporal classification performance.
• The proposed online learning method is suitable for power- and area-constrained neuromorphic processors.
• This approach enables on-device learning and reduces memory requirements for SNNs.