Choice Type Impacts Human Reinforcement Learning

Area of Science:

• Cognitive Science
• Neuroscience
• Computational Psychology

Background:

• Reinforcement learning (RL) models estimate stimulus-response values for incremental learning.
• RL models treat all response types uniformly, from concrete motor actions to abstract choices.
• The impact of choice type on the learning process remains underexplored.

Purpose of the Study:

• To investigate whether the learning process in reinforcement learning (RL) differs based on the type of choice.
• To identify the underlying mechanisms contributing to potential differences in learning speed and accuracy.

Main Methods:

• Experiment 1: Compared learning of concrete motor actions versus general choices using behavioral measures (speed, accuracy) and computational modeling.
• Experiment 2: Replicated findings and further investigated the role of working memory and RL speed.
• Ruled out task difficulty or complexity as confounding factors in both experiments.

Main Results:

• Participants exhibited slower and less accurate learning for general choices compared to concrete motor actions.
• Computational modeling revealed irrelevant credit assignment and slower information integration for general choices.
• Experiment 2 confirmed that weakened working memory, not slowed RL, accounted for the learning deficit in general choice conditions.

Conclusions:

• The type of choice significantly influences reinforcement learning (RL) processes.
• Abstract choice spaces recruit executive resources, potentially hindering efficient credit assignment and information integration.
• Understanding choice abstraction is crucial for optimizing learning in complex environments and developing more nuanced RL models.