Generalization guides human exploration in vast decision spaces

Area of Science:

• Cognitive Science
• Decision Science
• Reinforcement Learning

Background:

• Human behavior in complex environments is often a search problem.
• Optimal solutions are difficult to find under finite search horizons.
• Intelligent exploration of unobserved actions is crucial for navigating vast problem spaces.

Purpose of the Study:

• To investigate how humans search for rewards in large problem spaces.
• To understand the role of generalization in guiding human search strategies.
• To model human search behavior using computational approaches.

Main Methods:

• Utilized bandit tasks with up to 121 arms to simulate search problems.
• Incorporated spatial correlation of rewards in both generated and natural environments.
• Tested various probabilistic and heuristic models against human performance data.

Main Results:

• Gaussian process function learning combined with optimistic upper confidence bound (UCB) sampling robustly explains human search behavior.
• This model accurately captures how humans use generalization to guide exploration.
• Model parameters are recoverable and can simulate human-like search performance.

Conclusions:

• Gaussian process function learning with UCB sampling provides a powerful framework for understanding human exploration in complex environments.
• Generalization is a key mechanism enabling efficient search under uncertainty.
• The developed model offers insights into human decision-making and can simulate human-like performance.