Dopamine transients encode reward prediction errors independent of learning rates

Area of Science:

• Neuroscience
• Computational Psychiatry
• Reinforcement Learning

Background:

• Reinforcement learning theories propose dopamine encodes reward prediction errors (RPEs) scaled by a learning rate.
• Corticostriatal synaptic plasticity, modulated by dopamine, is believed to update value representations.
• This framework implies dopamine release reflects the product of RPEs and learning rates.

Purpose of the Study:

• To investigate how dopamine encodes learning rates in the nucleus accumbens core (NAcc) within a volatile environment.
• To determine if dopamine release reflects both RPEs and dynamic learning rates.

Main Methods:

• Rats performed a task with semi-observable states and varying rewards.
• Behavioral analyses examined trial initiation speed as a function of RPEs.
• Computational modeling assessed learning rates and Bayesian inference of hidden states.
• Dopamine release in the NAcc was measured during task performance.

Main Results:

• Rats adjusted trial initiation speeds based on RPEs, reflecting dynamic learning rates.
• Learning rates increased after state transitions and scaled with belief updates about hidden states.
• Dopamine release in the NAcc encoded RPEs independently of the calculated learning rates.

Conclusions:

• Dopamine in the NAcc encodes reward prediction errors, but not learning rates, in a volatile environment.
• Evidence suggests dopamine-independent mechanisms are responsible for instantiating dynamic learning rates.
• Findings challenge the traditional view of dopamine's role in scaling learning rates.