A reinforcement learning framework for spiking networks with dynamic synapses.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Traditional synaptic plasticity models often focus on weight updates.
• Integrating Hebbian learning with reinforcement learning (RL) offers a more comprehensive approach to synaptic dynamics.
• Dynamic synapses require mechanisms that go beyond simple weight modifications.

Purpose of the Study:

• To present a novel framework integrating Hebbian-based and RL rules for dynamic synapses.
• To enable Hebbian learning to modify synaptic model parameters, not just weights, using RL reward signals.
• To demonstrate the framework's efficacy in a spiking neural network for temporal computation.

Main Methods:

• A framework was developed where Hebbian rules update hidden synaptic model parameters based on RL temporal difference rewards.
• A spiking neural network with spike-timing-dependent synapses was employed.
• The network learned the exclusive-OR computation using temporally coded inputs and outputs.

Main Results:

• The network successfully learned the exclusive-OR computation, demonstrating effective temporal processing.
• The proposed framework successfully integrated Hebbian and RL principles.
• The approach proved tractable, computationally inexpensive, and applicable to various synaptic models and neural representations.

Conclusions:

• The integrated Hebbian and RL framework offers a powerful and efficient method for dynamic synaptic plasticity.
• This approach enhances the utility of biologically plausible synaptic models in signal processing.
• The framework's generality supports its application in diverse computational neuroscience and AI tasks.