Assisting Training of Deep Spiking Neural Networks With Parameter Initialization.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Spiking neural networks (SNNs) offer advantages in information encoding, computation, and power efficiency.
• Effective training of SNNs is hindered by challenges in weight distribution initialization.
• Backpropagation through time (BP) during initial training phases significantly impacts gradient generation.

Purpose of the Study:

• To address the critical problem of weight distribution initialization in SNNs.
• To develop an initialization technique that mitigates gradient vanishing during SNN training.
• To improve the accuracy and training speed of SNNs through optimized initialization.

Main Methods:

• Derivation of an asymptotic formula to approximate spiking neuron response curves.
• Development of a novel initialization technique based on the derived slant asymptote.
• Validation of the proposed method on MNIST and CIFAR10 classification tasks.

Main Results:

• The proposed initialization strategy significantly accelerates SNN training.
• The method demonstrably improves model accuracy compared to existing initialization techniques.
• Testing across diverse neuron configurations and hyperparameters confirms the method's versatility and superiority.

Conclusions:

• The slant asymptote-based initialization technique is effective for SNN training.
• This approach enhances both training speed and classification accuracy in SNNs.
• Recommendations for SNN training are provided based on the study's findings.