When models matter: Environmental demand guides the arbitration between model-based and model-free control.

Area of Science:

• Cognitive Psychology
• Neuroscience
• Reinforcement Learning

Background:

• Human decision-making involves both habitual (model-free) and goal-directed (model-based) control.
• Interindividual differences in strategy use are known, but adaptability is less understood.

Purpose of the Study:

• To investigate if humans can learn to regulate model-free versus model-based control based on environmental demands.
• To determine if participants adapt their strategy based on the frequency of environmental state alternations or repetitions.

Main Methods:

• A two-stage decision-making task was employed with 140 participants.
• Environmental demand was manipulated by varying the frequency of first-stage state alternations versus repetitions.
• Model-free behavior was indicated by repeating rewarded choices; model-based behavior by generalizing rewards across states.

Main Results:

• Participants exposed to more first-stage state alternations exhibited more model-based control in a subsequent test phase.
• Conversely, participants exposed to more first-stage state repetitions showed more model-free behavior.
• This suggests a learned arbitration between strategies.

Conclusions:

• Humans can adapt their reinforcement-learning strategy (model-free vs. model-based) in response to changing environmental demands.
• This adaptability appears to be a cost-benefit analysis sensitive to the environment.
• Findings highlight the dynamic nature of decision-making control.