SD²-SNN: Self-distillation and structural decomposition framework for SNNs in continual learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Catastrophic forgetting is a major challenge in Artificial Neural Networks (ANNs) for continual learning.
• Existing ANN methods for continual learning often increase computational overhead.
• Spiking Neural Networks (SNNs) offer energy efficiency but lack unified continual learning mechanisms.

Purpose of the Study:

• To propose SD²-SNN, a framework for SNNs to mitigate catastrophic forgetting without external supervision.
• To enhance knowledge retention in SNNs for long sequences of tasks.
• To balance plasticity and stability in SNNs for energy-efficient continual learning.

Main Methods:

• Implemented a Self-Distillation mechanism to anchor decision boundaries by aligning spike-rate distributions.
• Employed Structural Decomposition to create stable shared and dynamic task-specific parameters.
• Leveraged inherent SNN sparsity for efficient computation.

Main Results:

• SD²-SNN achieved strong and stable performance on image-based and event-based continual learning benchmarks.
• Demonstrated high accuracy on Split-CIFAR100, Tiny-ImageNet, and DVS128 Gesture datasets.
• Effectively balanced plasticity and stability, outperforming existing methods.

Conclusions:

• SD²-SNN offers an effective and energy-efficient solution for continual learning in SNNs.
• The proposed framework successfully mitigates catastrophic forgetting without external supervision.
• SD²-SNN paves the way for more robust and sustainable AI systems.