A cholinergic feedback circuit to regulate striatal population uncertainty and optimize reinforcement learning.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Cognitive Science

Background:

• The basal ganglia are crucial for reinforcement learning, adjusting action values based on reward prediction errors.
• Adaptive behavior in stochastic environments necessitates dynamic learning rate adjustments informed by uncertainty.

Purpose of the Study:

• To investigate the role of cholinergic tonically active interneurons (TANs) in the striatum for adaptive reinforcement learning.
• To model how TANs enable the consideration of uncertainty to dynamically adjust learning rates.

Main Methods:

• Development of computational models across three levels of analysis (Marr's framework).
• Neural modeling of TANs modulating spiny neuron excitability and population responses.
• Implementation of a feedback control system for dynamic modulation of TAN pauses by uncertainty.

Main Results:

• TANs modulate the effective learning rate by influencing spiny neuron excitability and population responses.
• Long TAN pauses enhance robustness to spurious outcomes by increasing synaptic weight divergence.
• Short TAN pauses promote stochastic behavior and responsiveness to contingency changes.
• A feedback system allows TAN pauses to be modulated by uncertainty, optimizing performance.

Conclusions:

• TANs provide a mechanism within the striatum for adaptive reinforcement learning in stochastic environments.
• Dynamic modulation of TAN activity based on uncertainty is key to optimizing behavioral flexibility.
• This mechanism supports efficient learning and decision-making under varying environmental conditions.