A simple computational algorithm of model-based choice preference.

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Behavioral Economics

Background:

• Reinforcement learning models often assume parallel model-free and model-based systems.
• The interplay between these systems in choice preference learning requires further investigation.

Purpose of the Study:

• To propose and test alternative computational models of choice learning.
• To investigate a sequential learning framework where model-free learning is primary and model-based information acts as a modulator.

Main Methods:

• Developed modified temporal-difference learning models with sequential processing assumptions.
• Compared proposed models against a standard parallel processing model using the two-stage decision task.
• Analyzed choice data from 23 human participants.

Main Results:

• Proposed sequential models, particularly the eligibility adjustment model, showed a better fit to participant choice data.
• The eligibility adjustment model offers a simpler algorithm and explains choices under both model-free and model-based control.
• Forgetting and variation models also improved data fits, supporting a hybrid approach.

Conclusions:

• A hybrid computational model integrating sequential processing best explains choice learning.
• This model captures individual differences in learning and exploration.
• Findings offer new insights into the interaction between model-free and model-based learning components.