Adaptive Synaptic Scaling in Spiking Networks for Continual Learning and Enhanced Robustness

Area of Science:

• Computational Neuroscience
• Artificial Intelligence

Background:

• Synaptic plasticity is crucial for neural network function, with synaptic scaling maintaining homeostasis.
• Spiking neural networks (SNNs) utilize backpropagation through time, but lack robust synaptic scaling mechanisms.

Purpose of the Study:

• To propose and evaluate an experience-dependent adaptive synaptic scaling mechanism (AS-SNN) for SNNs.
• To enhance SNN performance in perturbation resistance, continual learning, and graph learning tasks.

Main Methods:

• Developed a two-stage learning process: adaptive short-term potentiation/depression in the forward path and gradient-regulated long-term consolidation in the backward path.
• The mechanism uses presynaptic activity to modulate synaptic strength, theoretically proven to converge.
• Tested on N-MNIST benchmark for perturbation resistance and continual learning, and on graph learning tasks.

Main Results:

• AS-SNN improved accuracy by 44% on perturbation resistance and 25% on continual learning tasks on the N-MNIST benchmark.
• Observed expected firing rate callback and sparse coding in graph learning tasks.
• Demonstrated effectiveness and efficiency through ablation studies and cost evaluations.

Conclusions:

• The proposed nonparametric adaptive scaling method is effective and efficient for SNNs.
• AS-SNN shows significant potential for advancing continual and robust learning in SNNs.