Credit assignment to state-independent task representations and its relationship with model-based decision making.

Area of Science:

• Cognitive Neuroscience
• Reinforcement Learning
• Decision Science

Background:

• Model-free learning relies on experience without environment structure knowledge.
• It's assumed model-free systems use outcome-relevant features for state representation.
• This study investigates credit assignment in model-free systems.

Purpose of the Study:

• To challenge the assumption that model-free learning uses only outcome-relevant features.
• To investigate if model-free systems assign credit to irrelevant task representations.
• To explore the impact of irrelevant value associations on decision-making.

Main Methods:

• Analysis of data from 769 individuals performing a 2-step reward decision task.
• Examining reward prediction based on stimulus identity versus spatial-motor aspects.
• Investigating individual differences in goal-directed (model-based) strategy deployment.

Main Results:

• Participants assigned value to spatial-motor representations, which were outcome-irrelevant.
• These spatial-motor value associations influenced behavior across task features and stages.
• The impact of irrelevant value was reduced in individuals using more goal-directed strategies.

Conclusions:

• Model-free systems may assign credit to representations irrelevant to task outcomes.
• This challenges current understanding of model-free representation formation.
• Findings suggest environmental structure influences how representations are formed and regulated.