KLIF: An Optimized Spiking Neuron Unit for Tuning Surrogate Gradient Function.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Spiking neural networks (SNNs) offer advantages in temporal processing, low power, and biological realism.
• Training SNNs effectively, especially using surrogate gradient methods, faces challenges due to the sensitivity to surrogate gradient function shape.
• Current methods often use static surrogate gradient shapes, limiting adaptability during training.

Purpose of the Study:

• To introduce a novel spiking neural model, the k-based leaky integrate-and-fire (KLIF) model.
• To enable dynamic adjustment of surrogate gradient parameters during SNN training.
• To investigate the impact of a learnable parameter on SNN training and inference accuracy.

Main Methods:

• Developed the k-based leaky integrate-and-fire (KLIF) spiking neural model with a learnable parameter.
• Implemented KLIF to dynamically adjust the height and width of the surrogate gradient near the threshold.
• Evaluated KLIF on static (CIFAR-10, CIFAR-100) and neuromorphic (CIFAR10-DVS, DVS128-Gesture) datasets.

Main Results:

• The KLIF model demonstrated superior performance compared to the standard leaky integrate-and-fire (LIF) model.
• Performance improvements were observed across various datasets and network architectures.
• Dynamic adjustment of surrogate gradients via KLIF significantly impacts final training accuracy.

Conclusions:

• The KLIF model offers a significant advancement in training spiking neural networks.
• Dynamic surrogate gradient adjustment is crucial for optimizing SNN performance.
• KLIF presents a viable and potentially superior replacement for the LIF model in diverse SNN applications.