Implicit Counterfactual Effect in Partial Feedback Reinforcement Learning: Behavioral and Modeling Approach.

Area of Science:

• Cognitive psychology
• Neuroscience
• Decision-making

Background:

• Context significantly impacts learning by altering option values based on available choices.
• Complete feedback (showing chosen and unchosen outcomes) enhances contextual effects through outcome comparison.
• The emergence of contextual effects under partial feedback (without explicit counterfactuals) remains unclear.

Purpose of the Study:

• To investigate how contextual effects in learning emerge under partial feedback conditions.
• To model the learning mechanisms influenced by contextual information.
• To determine if contextual effects extend beyond probabilistic rewards to magnitude rewards.

Main Methods:

• Utilized partial and complete feedback paradigms with varying reward distributions.
• Employed computational modeling to analyze behavioral data.
• Compared model performance in accounting for learning behavior.

Main Results:

• A model where the chosen option's outcome updates both chosen and unchosen option values in opposing directions best explains the behavioral data.
• This finding aligns with the proposed diffusive effect of dopamine in the striatum.
• Contextual effects were observed for both probabilistic and magnitude rewards.

Conclusions:

• The contextual effect in learning can be explained by extending the concept of counterfactuals to include the influence of the chosen outcome on the unchosen option.
• This extended counterfactual framework accounts for learning even when explicit information about unchosen outcomes is absent.
• The study demonstrates the broad applicability of contextual learning effects across different reward structures.