Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Reinforcement Learning

Background:

• Midbrain dopamine neurons are traditionally modeled as signaling reward prediction errors in temporal difference (TD) learning.
• Standard TD models primarily use cached values and may not fully capture the complexity of real-world behavior and learning.
• Inferred, model-based value is crucial for many behaviors but is often excluded from standard dopamine signaling models.

Purpose of the Study:

• To investigate whether dopamine neurons signal errors related to inferred, model-based values.
• To determine if dopamine neurons integrate both cached and inferred values within a unified framework.
• To explore the extent to which dopamine neuron activity aligns with and deviates from standard TD learning predictions.

Main Methods:

• Electrophysiological recordings of dopamine neuron activity in rats.
• Experimental design incorporating cues with both directly experienced (cached) and inferred values.
• Analysis of dopamine neuron firing patterns in response to these different types of cue values and prediction errors.

Main Results:

• Error-signaling dopamine neurons in rats responded to the inferred, model-based value of novel cues.
• This response to inferred value occurred within the same framework used to track cached values of previously rewarded cues.
• Dopamine neuron activity demonstrated sensitivity to value derived through inference, not just direct experience.

Conclusions:

• Dopamine neurons appear to access and process a wider range of information than encompassed by standard TD models.
• The findings challenge the notion that dopamine neurons exclusively signal errors derived from TD predictions.
• This suggests a more sophisticated role for dopamine in learning, incorporating model-based inference alongside simpler associative learning.