Short-term memory traces for action bias in human reinforcement learning.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Cognitive Science

Background:

• Reinforcement learning models how organisms learn from rewards and punishments.
• The credit assignment problem is central to reinforcement learning, concerning delayed rewards.
• Eligibility traces (ETs) enhance temporal difference learning by remembering past choices.

Purpose of the Study:

• To investigate if reinforcement learning models with long-lasting eligibility traces (ETs) can better explain human behavior.
• To examine the impact of timing between choices on human performance in a sequential economic decision game.

Main Methods:

• Human subjects played a sequential economic decision game with differing short-term and long-term optimal strategies.
• Behavioral data was analyzed to assess the influence of inter-choice intervals on performance.
• A temporal difference learning model incorporating ETs was used to explain observed human behavior.

Main Results:

• Human performance was significantly affected by the time between choices, in a counterintuitive manner.
• This behavioral pattern was accurately predicted by a temporal difference learning model with persistent ETs.
• Recent findings suggest short-term synaptic plasticity in dopamine neurons could mechanistically support these ETs.

Conclusions:

• Reinforcement learning models incorporating eligibility traces that persist across actions provide a compelling explanation for human decision-making in sequential tasks.
• The findings bridge computational models of learning with neurobiological mechanisms, highlighting dopamine neuron plasticity.
• This work advances our understanding of the neural underpinnings of learning and decision-making under delayed reward conditions.