What to Choose Next? A Paradigm for Testing Human Sequential Decision Making.

Area of Science:

• Cognitive Science
• Neuroscience
• Computational Psychology

Background:

• Everyday decisions are often sequential with infrequent rewards.
• Reinforcement learning models explore sequential decision-making theoretically.
• A systematic experimental paradigm for testing these models was lacking.

Purpose of the Study:

• To develop a systematic experimental paradigm for sequential decision-making.
• To investigate key components of reinforcement learning models in humans.
• To compare model-free and model-based learning mechanisms in human behavior.

Main Methods:

• Developed a novel experimental paradigm for sequential decision-making.
• Measured human participants' performance in tasks with sparse rewards.
• Analyzed the decay of eligibility traces (memory traces) based on decision steps and time.
• Assessed the influence of monetary rewards and environmental structure on performance.

Main Results:

• Eligibility traces decay with the number of decision steps, not elapsed time.
• Monetary rewards and environmental spatial regularity did not significantly impact performance.
• Model-free learning algorithms provided a better fit to human behavior than model-based algorithms.

Conclusions:

• The developed paradigm effectively probes sequential decision-making and reinforcement learning components.
• Human memory traces in decision-making are sensitive to the number of steps, not just time.
• Model-free learning mechanisms appear dominant in human sequential decision-making under these conditions.