The ubiquity of model-based reinforcement learning

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Psychiatry

Background:

• The reward prediction error (RPE) theory, based on model-free reinforcement learning (RL), explains dopamine's role in learning and decision-making.
• Model-based RL methods offer more flexible decision-making by using internal world models to predict future outcomes.
• Recent findings suggest model-based computations overlap with brain regions previously associated with model-free learning.

Purpose of the Study:

• To review behavioral and neural evidence concerning model-free and model-based learning systems.
• To reconcile the apparent cohabitation of these two systems in the brain.
• To advance understanding of dopamine's function in complex decision-making.

Main Methods:

• Review of existing behavioral studies on reinforcement learning.
• Analysis of neuroimaging data investigating neural correlates of learning.
• Synthesis of evidence from computational neuroscience models.

Main Results:

• Evidence supports the presence of both model-free and model-based learning mechanisms in the brain.
• Neural signatures of model-based computations are found in areas traditionally linked to model-free RL.
• The precise interaction and segregation of these systems remain under investigation.

Conclusions:

• The brain likely employs both model-free and model-based strategies for learning and decision-making.
• Reconciling the overlapping neural substrates of these systems is crucial for understanding dopamine function.
• Future research should focus on elucidating the mechanisms underlying the integration or segregation of these learning systems.