When Does Model-Based Control Pay Off?

Area of Science:

• Cognitive Science
• Neuroscience
• Computational Psychology

Background:

• Decision-making research often proposes two systems: fast, automatic (model-free) and slow, deliberative (model-based).
• Model-free strategies are quick but potentially inaccurate; model-based strategies are accurate but cognitively demanding.
• A trade-off between accuracy and computational demand is assumed to govern strategy selection.

Purpose of the Study:

• To investigate the accuracy-demand trade-off in model-free versus model-based reinforcement learning strategies.
• To identify factors limiting the effectiveness of model-based strategies in standard tasks.
• To develop and validate a novel task that establishes a clear accuracy-demand trade-off.

Main Methods:

• Analysis of standard tasks used to differentiate model-free and model-based strategies.
• Identification of five factors diminishing model-based strategy effectiveness.
• Design and implementation of a modified task to create a formal accuracy-demand trade-off.
• Empirical testing with human participants to observe strategy reliance.

Main Results:

• Standard tasks do not effectively embody an accuracy-demand trade-off between model-free and model-based strategies.
• Five identified factors reduce model-based strategy accuracy and relevance.
• The novel task design successfully creates a formal and empirical accuracy-demand trade-off.
• Human participants showed increased reliance on model-based control in the new task.

Conclusions:

• The assumed accuracy-demand trade-off in decision-making tasks is not inherent in standard paradigms.
• A revised task design can elicit a genuine trade-off, influencing strategy selection.
• This work provides a valuable tool for studying how humans balance cognitive demands and accuracy in choice.