Training spiking neuronal networks to perform motor control using reinforcement and evolutionary learning

Area of Science:

• Computational Neuroscience
• Machine Learning

Background:

• Spiking neural networks (SNNs) show promise for sensory-motor behaviors but often underperform compared to artificial neural networks (ANNs).
• Optimizing SNN training requires exploring diverse learning mechanisms that mimic biological neural processes.

Purpose of the Study:

• To compare the efficacy of two distinct learning mechanisms, spike-timing-dependent reinforcement learning (STDP-RL) and evolutionary strategy (EVOL), for training SNNs.
• To advance the performance of SNNs in reinforcement learning (RL) tasks.

Main Methods:

• Trained SNNs to solve the CartPole control problem using both STDP-RL and EVOL learning mechanisms.
• Compared the performance of SNNs trained with each method.

Main Results:

• EVOL demonstrated superior performance in training SNNs for the CartPole task compared to STDP-RL.
• EVOL offers a viable alternative to STDP-RL, potentially by not requiring the explicit modeling of all interacting synaptic plasticity components.

Conclusions:

• Evolutionary strategy is a powerful and effective method for training spiking neural networks to perform complex sensory-motor behaviors.
• This work enhances the applicability of SNNs in reinforcement learning and provides a platform for studying multi-timescale neural learning mechanisms.