Reinforcement learning, spike-time-dependent plasticity, and the BCM rule

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Learning agents require parameter updates for behavioral improvement.
• Reinforcement learning utilizes environmental reward signals to guide these updates.
• Spiking neural networks offer a biologically plausible model for computation.

Purpose of the Study:

• To derive a spike-time-dependent plasticity rule for spiking neural networks using a machine learning policy algorithm.
• To ensure convergence to a local optimum of the expected average reward.
• To demonstrate the rule's applicability to various neuronal models and its relation to biological plasticity.

Main Methods:

• Application of a recent machine learning policy learning algorithm to spiking neural networks.
• Derivation of a spike-time-dependent plasticity (STDP) rule.
• Testing the rule on several computational 'toy problems'.
• Statistical analysis to compare the derived rule with existing models.

Main Results:

• A novel STDP rule was successfully derived for spiking neural networks.
• The rule guarantees convergence towards an optimal average reward.
• The derived rule is compatible with complex neuronal models like the Hodgkin-Huxley model.
• The new rule shows strong statistical similarity to the biologically supported BCM rule.

Conclusions:

• The developed STDP rule provides a principled way to train spiking neural networks for reinforcement learning tasks.
• This work bridges machine learning policy optimization with biologically realistic neural computation.
• The findings suggest potential mechanisms for synaptic plasticity in biological systems and offer a powerful tool for AI development.