A scalable implementation of the recursive least-squares algorithm for training spiking neural networks

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Spiking recurrent neural networks (SNNs) are increasingly used to model neural computations.
• Training large-scale SNNs on complex data is computationally intensive and time-consuming.
• Efficient training algorithms are crucial for advancing neuroscience research.

Purpose of the Study:

• To present optimized CPU and GPU implementations of the recursive least-squares algorithm for training SNNs.
• To significantly reduce the time and resources required for training large SNN models.
• To enable more interactive and real-time computational neuroscience studies.

Main Methods:

• Developed optimized CPU and GPU implementations of the recursive least-squares algorithm.
• Tested the implementations on large-scale SNNs with millions of neurons and billions of synapses.
• Validated the approach by training a network to reproduce extensive neuronal recordings from a mouse decision-making task.

Main Results:

• The GPU implementation achieved training speeds approximately 1,000 times faster than an unoptimized CPU reference.
• Successfully trained a large network (>66,000 neurons) to replicate mouse brain activity in under an hour.
• Demonstrated the scalability of the method for networks with up to one million neurons.

Conclusions:

• Optimized recursive least-squares algorithms provide a significant speedup for training SNNs.
• Enables interactive in-silico studies of complex neural dynamics and multi-area computations.
• Facilitates real-time model training concurrent with in-vivo experiments, bridging computational and experimental neuroscience.