Learning Reward Uncertainty in the Basal Ganglia

Area of Science:

• Neuroscience
• Computational Neuroscience
• Decision Neuroscience

Background:

• Optimal decision-making relies on learning reward reliability.
• The basal ganglia are known to learn expected reward, but not reward uncertainty.
• Understanding reward uncertainty estimation is key to explaining complex choices.

Purpose of the Study:

• To present a novel class of computational models for reward uncertainty estimation in the basal ganglia.
• To elucidate how neural circuits encode both mean reward and reward spread.
• To investigate the role of dopamine in modulating risk-seeking behavior.

Main Methods:

• Developed computational models of basal ganglia circuits.
• Modeled synaptic weight differences and sums in D1 and D2 neurons to represent mean and spread.
• Simulated the effects of varying tonic dopamine levels on choice behavior.

Main Results:

• The models successfully encode both mean reward and reward spread.
• Synaptic weight differences in D1/D2 neurons represent mean reward, while their sum represents spread.
• Adjusting dopamine levels in the models altered the tendency towards risk-seeking or risk-averse choices.

Conclusions:

• The proposed models offer a framework for understanding reward uncertainty processing in the basal ganglia.
• Dopamine acts as a modulator, controlling choices involving variable rewards.
• The models align with basal ganglia physiology, explain dopaminergic effects on risk, and generate testable predictions.