Reinforcement learning: computing the temporal difference of values via distinct corticostriatal pathways

Area of Science:

• Neuroscience
• Computational Neuroscience
• Systems Neuroscience

Background:

• Midbrain dopamine neurons are implicated in reward prediction error (RPE) signaling.
• The precise neural mechanisms underlying RPE computation remain largely unknown.

Purpose of the Study:

• To propose a novel computational mechanism for RPE signaling in corticostriatal circuits.
• To explain how distinct neuronal subpopulations contribute to representing temporal differences in value.

Main Methods:

• Theoretical modeling based on recent findings in corticostriatal circuits.
• Hypothesizing specific connectivity patterns and neuronal dynamics.

Main Results:

• Two distinct corticostriatal neuron subpopulations differentially encode current and previous states/actions.
• Unidirectional connectivity and recurrent excitation within subpopulations are key.
• Selective connections to basal ganglia direct and indirect pathways enable temporal difference computation.

Conclusions:

• This model provides a unified framework for understanding basal ganglia functions.
• The proposed mechanism offers insights into the neural basis of RPE and has potential clinical implications for neurological and psychiatric disorders.