A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Biology

Background:

• The boundary model explains decision-making but not how boundaries are learned.
• Optimizing decision boundaries is crucial for efficient behavior under uncertainty.

Purpose of the Study:

• To propose a model-free reinforcement learning algorithm for perceptual decisions under uncertainty.
• To investigate how animals learn and optimize decision boundaries during sequential information sampling.

Main Methods:

• Developed a reinforcement learning algorithm integrating sequential sampling with an implicit decision boundary.
• Simulated the model to reproduce key features of perceptual decision-making.

Main Results:

• The model successfully reproduced canonical features of perceptual decision-making, including accuracy and reaction time dependencies.
• Demonstrated the model's ability to modulate the speed-accuracy trade-off based on payoff regimes.
• Showcased the model's capacity to learn optimal strategies for committing to decisions or continuing information sampling.

Conclusions:

• The proposed framework unifies learning and decision-making, offering insights into behavioral flexibility.
• This model provides a novel perspective on the mechanisms underlying context-dependent behavioral adjustments.