Dissociation between asymmetric value updating and perseverance in human reinforcement learning

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Behavioral Economics

Background:

• The learning rate in reinforcement learning dictates how new information updates actions.
• Existing research suggests human learning rates are biased by reward prediction error sign.
• This observed asymmetry may be a statistical artifact due to unmodeled choice autocorrelation (perseverance).

Purpose of the Study:

• To develop and validate a Hybrid model that disentangles asymmetric learning rates from perseverance in human choice behavior.
• To investigate the genuine drivers of human decision-making in reinforcement learning tasks.
• To re-evaluate previous findings of asymmetric learning in existing datasets.

Main Methods:

• Simulations were conducted to confirm the Hybrid model's ability to accurately identify underlying processes.
• Empirical data from a web-based experiment were analyzed using the Hybrid model.
• The Hybrid model was applied to two previously published datasets.

Main Results:

• Simulations demonstrated the Hybrid model's efficacy in distinguishing between asymmetric learning and perseverance.
• Analysis of web-based experiment data indicated that perseverance, not asymmetric learning, primarily governed choice behavior.
• Re-analysis of two open datasets confirmed asymmetric learning in one but not the other.

Conclusions:

• Perseverance is a critical factor in human reinforcement learning that must be accounted for.
• The Hybrid model provides a more accurate framework for understanding human choice behavior than models solely focused on learning rates.
• Future research should consider both learning asymmetry and perseverance when modeling decision-making.