Adaptive learning rate of SpikeProp based on weight convergence analysis

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Spiking Neural Networks (SNNs) training with SpikeProp can suffer from learning cost surges.
• These surges disrupt learning and can lead to training failure.
• Appropriate learning rates are essential for mitigating these surges.

Purpose of the Study:

• To develop an adaptive learning rate extension for SpikeProp.
• To ensure the convergence of the SNN learning process.
• To improve weight convergence and learning speed.

Main Methods:

• Performed weight convergence analysis to determine optimal step sizes for weight updates.
• Derived an adaptive learning rate extension to the SpikeProp algorithm.
• Analyzed performance via simulations on various benchmarks comparing with existing methods.

Main Results:

• The adaptive learning rate significantly improves weight convergence.
• The proposed method accelerates the learning process in SNNs.
• Simulations demonstrated superior performance compared to existing approaches.

Conclusions:

• An adaptive learning rate is crucial for stable and efficient SNN training.
• The derived extension effectively prevents learning surges and ensures convergence.
• This approach offers a significant advancement in training Spiking Neural Networks.