Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free...

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• Behavioral control allocation between model-based and model-free reinforcement-learning (RL) systems is influenced by their relative reliability.
• The impact of task complexity on this arbitration process remains poorly understood.

Purpose of the Study:

• To investigate how task complexity, alongside state-space uncertainty, affects the arbitration between model-based and model-free RL strategies.
• To explore the neural underpinnings of this interaction using computational fMRI.

Main Methods:

• Novel task design incorporating varying levels of task complexity and state-space uncertainty.
• Computational modeling to analyze behavioral choices.
• Model-based functional magnetic resonance imaging (fMRI) to examine neural activity.

Main Results:

• Participants increased reliance on model-based RL with rising task complexity.
• When both task complexity and uncertainty were high, participants shifted towards model-free RL.
• Computational fMRI identified an interaction between task complexity and neural representations of RL system reliability in the inferior prefrontal cortex.

Conclusions:

• Task complexity and state-space uncertainty interact to modulate the arbitration between model-based and model-free reinforcement-learning.
• The inferior prefrontal cortex plays a role in integrating information about task demands and the reliability of different learning systems.