Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Machine learning

Background:

• Existing reinforcement learning (RL) models often overlook the complexity of synaptic plasticity in neural systems.
• Current RL models with spiking neurons typically incorporate only a single plasticity mechanism.

Purpose of the Study:

• To propose a neural realistic reinforcement learning (RL) model that integrates and coordinates multiple synaptic plasticity mechanisms.
• To enhance the learning capabilities of RL models by incorporating biological realism.

Main Methods:

• Developed a novel RL model coordinating stochastic and deterministic synapse plasticities.
• Stochastic synapse plasticity implemented via the hedonistic rule (modulating neurotransmitter release probability).
• Deterministic synapse plasticity implemented via a reward-modulated spike-timing-dependent plasticity (STDP) rule (modulating synaptic strengths).

Main Results:

• Evaluated the model on logic gate function learning and a 19-state random walk problem.
• Demonstrated that coordinating diverse synaptic plasticities leads to rapid and stable learning.
• The proposed model shows improved performance compared to single-plasticity mechanisms.

Conclusions:

• Coordination of diverse synaptic plasticities is crucial for effective and biologically plausible reinforcement learning.
• The proposed neural realistic RL model offers a promising direction for advancing AI and computational neuroscience.
• This approach enhances learning efficiency and stability in RL agents.