Dopamine, reward learning, and active inference.

Area of Science:

• Neuroscience
• Computational Psychiatry
• Reinforcement Learning

Background:

• Temporal difference learning models suggest dopamine signals reward prediction errors crucial for learning.
• Optogenetic stimulation of dopamine neurons can substitute for actual rewards, supporting this view.
• However, dopamine depletion often impairs performance rather than learning, creating a paradox.

Purpose of the Study:

• To resolve the paradox of dopamine's role in learning and performance.
• To propose and test a novel hypothesis where dopamine encodes action-outcome belief precision.
• To explain how dopamine controls the outcome-sensitivity of behavior.

Main Methods:

• Extending an active inference framework for Markov decision processes to incorporate learning.
• Simulating dopamine dynamics during instrumental conditioning.
• Modeling the effects of simulated dopamine depletion and excitation on learning and performance.

Main Results:

• Simulated dopamine dynamics closely mirrored observed experimental findings.
• Simulated dopamine depletion impaired task performance but preserved learning.
• Simulated dopamine neuron excitation led to reward learning via altered outcome state inference.

Conclusions:

• Dopamine's role in learning is reconciled by its function in encoding action-outcome belief precision.
• This framework explains divergent experimental findings on dopamine's necessity for learning.
• The model offers a parsimonious explanation for dopamine's influence on behavior and learning.