Dopamine neurons learn to encode the long-term value of multiple future rewards

Area of Science:

• Neuroscience
• Behavioral Economics
• Computational Neuroscience

Background:

• Midbrain dopamine neurons are crucial for signaling reward value, prediction errors, and event salience.
• Reinforcement learning theories suggest these neurons should encode long-term future rewards for goal-directed behavior.

Purpose of the Study:

• To experimentally investigate whether dopamine neurons encode long-term future rewards.
• To examine the role of dopamine neurons in multistep choice tasks involving delayed gratification.

Main Methods:

• Recorded activity of 185 dopamine neurons in three monkeys performing a multistep choice task.
• Analyzed anticipatory licking behavior as an indicator of reward expectation.
• Utilized reinforcement learning models to predict dopamine neuron responses.

Main Results:

• Monkeys' licking behavior indicated anticipation of the sum of immediate and multiple future rewards, not just immediate rewards.
• Dopamine neuron responses to cues and reinforcers reflected the expected values of multiple future rewards.
• Neuron responses quantitatively matched theoretical value functions incorporating time discounting.

Conclusions:

• Dopamine neurons learn to encode the long-term value of multiple future rewards.
• These findings support the role of dopamine in planning and executing actions toward distant goals.
• The study validates reinforcement learning models of dopamine function in decision-making.