Generalization of value in reinforcement learning by humans

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• Decision-making research often models choices using reinforcement learning, focusing on dopamine and striatal pathways.
• Basic reinforcement learning models do not fully explain how learning stimulus regularities or relations guides choices.
• The hippocampus is implicated in learning stimulus-stimulus relations, distinct from the striatum's stimulus-response learning.

Purpose of the Study:

• To investigate the combined roles of striatal reinforcement learning and hippocampal relational memory in decision-making.
• To determine if learning stimulus relations enhances reinforcement learning by enabling generalization.
• To explore how these systems interact during complex choice behavior.

Main Methods:

• Functional magnetic resonance imaging (fMRI) was used to measure brain activity.
• Computational model-based analyses compared basic reinforcement learning with an augmented model incorporating relational generalization.
• Participants performed a reinforcement learning task with an uninstructed correlated reward structure between option pairs.

Main Results:

• Both striatal and hippocampal activity correlated with learning during the task.
• Choice behavior and striatal blood oxygen-level-dependent (BOLD) activity were better explained by the augmented model that allowed for relational generalization.
• Functional connectivity between the ventral striatum and hippocampus was modulated by the model's ability to capture individual choice patterns.

Conclusions:

• Reinforcement learning in the striatum may utilize relational representations typically associated with the hippocampus.
• Decision-making involves an interactive system where memory and reinforcement learning mechanisms are integrated.
• This integrated model provides a more comprehensive account of how humans learn and make choices in complex environments.