Continuous learning of spiking networks trained with local rules

Area of Science:

• Computational Neuroscience
• Artificial Intelligence

Background:

• Artificial neural networks (ANNs) suffer from catastrophic forgetting (CF) when learning sequentially.
• Biological brains exhibit continuous learning without CF, offering a model for more robust AI.
• Spiking neural networks (SNNs), inspired by biological neural networks, are a potential solution for CF.

Purpose of the Study:

• To investigate the susceptibility of Spiking Neural Networks (SNNs) to catastrophic forgetting.
• To evaluate biologically inspired methods for mitigating CF in SNNs trained with local rules.

Main Methods:

• SNNs were trained using biologically plausible local training rules, specifically spike-timing-dependent plasticity (STDP).
• A novel method using stochastic Langevin dynamics was developed to assess synapse importance without global gradients.
• Comparative analysis included adapted CF prevention methods from ANNs and the novel Langevin dynamics approach.

Main Results:

• The study systematically analyzed CF in SNNs under biologically plausible training conditions.
• The developed Langevin dynamics method provided a gradient-free approach to synapse importance for CF mitigation.
• Performance was evaluated on public datasets within the SpykeTorch simulation environment.

Conclusions:

• SNNs trained with local STDP rules demonstrate susceptibility to catastrophic forgetting.
• Biologically inspired methods, including the novel Langevin dynamics approach, show potential for mitigating CF in SNNs.
• Further research into gradient-free CF prevention is crucial for advancing continuous learning in SNNs.