Putting bandits into context: How function learning supports decision making.

Area of Science:

• Cognitive Psychology
• Computational Neuroscience
• Machine Learning

Background:

• Decision-making under uncertainty is a fundamental cognitive process.
• The multi-armed bandit task is a common framework for studying reinforcement learning.
• Existing models often simplify the complexity of real-world decision environments.

Purpose of the Study:

• Introduce the contextual multi-armed bandit task to model learning in uncertain environments.
• Investigate how humans learn reward functions from contextual features.
• Compare human learning strategies to computational models.

Main Methods:

• Participants engaged in repeated choices within a contextual multi-armed bandit task.
• Contextual features predicted rewards via initially unknown functions.
• Behavioral data were analyzed and compared against context-blind and contextual learning models (e.g., Gaussian process).

Main Results:

• Participants generally learned the context-reward relationships.
• Human behavior was best described by a Gaussian process learning strategy, indicating generalization of experience.
• Learning strategies varied with task complexity, with some participants reverting to simpler, context-blind approaches in nonlinear environments.

Conclusions:

• The contextual multi-armed bandit task effectively captures human learning in uncertain environments.
• Gaussian process models provide a strong account of human generalization and adaptation in this task.
• Individual differences and task demands influence the adoption of sophisticated learning strategies.