One-shot learning and behavioral eligibility traces in sequential decision making.

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• Daily tasks involve sequential decision-making, requiring mechanisms to link actions to future rewards.
• Reinforcement learning (RL) addresses the credit assignment problem using temporal-difference learning or eligibility traces.
• Eligibility traces enable one-shot learning by reinforcing entire action sequences from single experiences.

Purpose of the Study:

• To investigate and provide direct evidence for one-shot sequence learning in reinforcement learning.
• To observe which actions and states are reinforced after a single reward in a multi-step sequence.
• To differentiate and identify behavioral and physiological signatures of RL with and without eligibility traces.

Main Methods:

• Development of a novel experimental paradigm to directly observe reinforcement of actions and states in sequences.
• Analysis focused on states where RL with and without eligibility traces predict distinct outcomes.
• Measurement of behavioral choices (choice probability) and physiological responses (pupil dilation).

Main Results:

• Direct behavioral evidence of reinforcement learning with eligibility traces was observed.
• Physiological signatures, including pupil dilation, also indicated reinforcement learning with eligibility traces.
• These findings were consistent across multiple sensory modalities.

Conclusions:

• The study provides direct empirical support for reinforcement learning models incorporating eligibility traces.
• One-shot learning of action sequences is behaviorally and physiologically observable.
• Eligibility traces play a significant role in efficient learning from single experiences in complex decision-making.