The "proactive" model of learning: Integrative framework for model-free and model-based reinforcement learning...

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• Reinforcement learning (RL) explains associative learning via reward signals.
• RL can be studied using model-based and model-free approaches.
• Distinct brain regions are implicated in each RL system.

Purpose of the Study:

• To propose the ventral striatum (VS) as the value function component of an RL agent.
• To integrate the proactive brain concept and default network into a model-based RL framework.
• To elucidate the neural mechanisms underlying reward expectation and integration in RL.

Main Methods:

• Conceptual framework integrating existing neuroscientific findings.
• Analysis of functional connectivity related to the VS.
• Application of the proactive brain concept to default network functions.

Main Results:

• The VS integrates model-free reward prediction errors and model-based inputs to compute value.
• The default network facilitates model-based RL by organizing environmental context frames.
• The orbitofrontal cortex (OFC) integrates reward information for expectation computation.

Conclusions:

• The VS functions as the value component in a proposed RL agent model.
• The default network supports model-based RL through contextual framing and prediction.
• OFC efferents to model-free structures facilitate the integration of model-based expectations into value signals.