Gradient estimation in dendritic reinforcement learning.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Synaptic plasticity is crucial for learning and memory.
• Complex neuronal models with dendritic zones allow for localized signaling.
• Reinforcement learning in neural networks often relies on simplified neuron models.

Purpose of the Study:

• To investigate synaptic plasticity rules in complex neuronal models.
• To compare learning performance between local and nonlocal plasticity mechanisms.
• To explore the role of dendritic NMDA-spikes in reinforcement learning.

Main Methods:

• Derivation of two plasticity rules: zone reinforcement (ZR) and cell reinforcement (CR).
• Simulation of a complex neuronal cell model capable of NMDA-spikes.
• Optimization of expected reward using stochastic gradient ascent for both rules.
• Comparison of learning performance between ZR and CR.

Main Results:

• Both ZR and CR optimize expected reward via stochastic gradient ascent.
• CR, utilizing nonlocal feedback from the soma, significantly enhances learning performance compared to ZR.
• Nonlocal feedback provides a distinct advantage for learning in complex neurons.

Conclusions:

• Nonlocal feedback mechanisms are critical for superior learning in complex neurons.
• Complex neurons offer computational advantages over simple point neurons due to nonlocal signaling.
• This study highlights the importance of neuronal morphology and dendritic integration in learning.