Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement...

Area of Science:

• Neuroscience
• Behavioral Science
• Computational Neuroscience

Background:

• Learning is often attributed to reward value prediction errors.
• However, learning can also occur when reward identity changes, even if value remains constant.
• This identity-based learning suggests a need for internal environmental models, challenging traditional model-free reinforcement learning.

Purpose of the Study:

• To investigate the neural mechanisms underlying value-based versus identity-based prediction error learning.
• To differentiate the roles of brain regions in learning from changes in reward value versus identity.
• To inform and potentially modify existing reinforcement learning models.

Main Methods:

• Utilized unblocking procedures in rats to assess learning.
• Trained rats to associate visual cues with specific food quantities and identities.
• Introduced novel auditory cues paired with changes in reward quantity or identity, followed by probe tests.

Main Results:

• The ventral striatum was essential for learning from changes in reward value, consistent with temporal difference reinforcement learning (TDRL) models.
• Both the ventral striatum and orbitofrontal cortex were required for learning from changes in reward identity.
• These findings indicate that identity-based learning relies on more than just value prediction errors.

Conclusions:

• Existing TDRL models need to be updated to incorporate model-based representations of outcome features for identity-based learning.
• The orbitofrontal cortex plays a crucial role in identity-based learning, requiring further delineation within these models.
• This research highlights the complexity of learning mechanisms beyond simple reward valuation.