Reinforcement learning and human behavior.

Area of Science:

• Cognitive Science
• Neuroscience
• Computational Psychiatry

Background:

• Model-free reinforcement learning (RL) is the primary computational framework for operant learning.
• Growing evidence suggests human operant learning is more complex than model-free RL can explain.
• Existing RL models struggle to account for all observed human and animal behaviors.

Purpose of the Study:

• To review current developments in reinforcement learning (RL) models for operant learning.
• To highlight the limitations of current RL approaches in explaining complex human behavior.
• To discuss the role of world models in RL and alternative policy-learning strategies.

Main Methods:

• Review of behavioral and neuroscientific evidence related to operant learning.
• Analysis of theoretical advances in reinforcement learning, including hierarchical and model-based RL.
• Discussion of alternative computational models for policy learning.

Main Results:

• Model-free RL alone cannot fully capture the nuances of operant learning.
• Hierarchical and model-based RL offer improved explanatory power for some behaviors.
• Certain aspects of human behavior remain unexplained by current RL paradigms.
• Learning a world model is crucial for, or concurrent with, policy learning in RL.

Conclusions:

• Current RL models require significant advancement to fully model human operant learning.
• Integrating world models into RL frameworks is essential for future progress.
• Alternative models that learn policies without explicit world models warrant further investigation.