Model-based learning and the contribution of the orbitofrontal cortex to the model-free world

Area of Science:

• Neuroscience
• Computational Neuroscience
• Behavioral Economics

Background:

• Learning is driven by discrepancies between predicted and received rewards.
• Two systems are proposed: model-free (cached values) and model-based (detailed predictions).
• The ventral striatum (VS) is key in model-free reinforcement learning, signaling reward predictions.

Purpose of the Study:

• To review evidence for model-based information processing in the VS and OFC.
• To propose how the orbitofrontal cortex (OFC) interacts with model-free circuitry.
• To explore the role of OFC in enhancing behavioral flexibility and learning.

Main Methods:

• Review of existing animal learning studies.
• Analysis of human neuroimaging data.
• Synthesis of current models of reinforcement learning.

Main Results:

• The VS signals both model-free and model-based information.
• The orbitofrontal cortex (OFC) is implicated in signaling model-based information.
• Evidence suggests OFC provides model-based signals to the traditional VS-dopamine pathway.

Conclusions:

• The OFC likely contributes model-based information to the established model-free reinforcement learning circuit.
• This interaction may underlie enhanced behavioral flexibility and learning from reward attributes.
• Further research is needed to elucidate the precise mechanisms of OFC-VS interaction in learning.