A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback

Area of Science:

• Computational Neuroscience
• Machine Learning
• Neural Networks

Background:

• Spiking neural networks (SNNs) require adaptive learning rules for complex behavior.
• Reward-modulated spike-timing-dependent plasticity (STDP) is a candidate for self-organizing learning in SNNs.
• Previous analysis of reward-modulated STDP relied solely on computer simulations.

Purpose of the Study:

• To develop analytical tools for reward-modulated STDP.
• To predict conditions under which reward-modulated STDP achieves desired learning outcomes.
• To explain how SNNs can solve credit assignment problems and adapt to temporal firing patterns.

Main Methods:

• Developed analytical framework for reward-modulated STDP.
• Modeled biofeedback experiment in monkeys using STDP and spontaneous firing.
• Analyzed network stability with STDP in recurrent neural networks.

Main Results:

• Analytical treatment predicts conditions for effective reward-modulated STDP.
• Neurons can learn to classify spatial and temporal firing patterns.
• Reward-modulated STDP can solve difficult credit assignment problems, explaining monkey biofeedback.
• Model provides functional explanation for trial-to-trial variability in cortical networks.
• STDP application to all synapses in recurrent networks does not compromise network stability.

Conclusions:

• Analytical tools enable prediction of reward-modulated STDP efficacy.
• Reward-modulated STDP offers a mechanism for complex learning in SNNs.
• The model explains biofeedback experiments and neuronal variability, advancing understanding of biological and artificial learning systems.