Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Neuroplasticity

Background:

• Spike-timing-dependent plasticity (STDP) modifies synaptic efficacy based on spike timing.
• Understanding how neural networks learn from reward signals is crucial for artificial intelligence and neuroscience.

Purpose of the Study:

• To derive and demonstrate learning rules based on reward-modulated STDP for reinforcement learning in spiking neural networks.
• To explore the efficacy of different reward-modulated STDP rules, including those with eligibility traces.

Main Methods:

• Analytical derivation of learning rules using a reinforcement learning algorithm applied to the stochastic spike response model.
• Simulations of integrate-and-fire neuron networks to test modulated STDP and modulated STDP with eligibility traces.
• Testing the rules on tasks like the XOR problem and learning target firing-rate patterns.

Main Results:

• Developed biologically plausible learning rules combining reward modulation with STDP and intrinsic plasticity.
• Demonstrated that modulated STDP rules can solve complex learning tasks, including those with delayed rewards.
• Showcased the ability of these rules to learn target output firing-rate patterns.

Conclusions:

• Reward-modulated STDP provides a viable mechanism for reinforcement learning in spiking neural networks.
• The proposed learning rules are versatile for training artificial spiking neural networks and suggest experimental investigations into reward-modulated STDP in biological systems.