Deconstructing the human algorithms for exploration.

Area of Science:

• Cognitive Science
• Neuroscience
• Machine Learning

Background:

• The exploration-exploitation dilemma is central to reinforcement learning.
• Human algorithms for resolving this dilemma remain unclear due to conflicting experimental data.

Purpose of the Study:

• To differentiate between two families of reinforcement learning algorithms based on uncertainty's effect on exploration.
• To investigate how humans navigate the exploration-exploitation trade-off.

Main Methods:

• Distinguishing algorithms by their prediction of uncertainty's impact on response bias versus slope.
• Conducting two experiments to observe behavioral changes.
• Employing computational modeling to validate findings.

Main Results:

• Evidence supports both uncertainty bonus (response bias) and sampling (response slope) algorithm predictions.
• Observed changes in both response bias and slope as a function of uncertainty.

Conclusions:

• Human exploration strategies are not monolithic and can be explained by a combination of algorithms.
• A hybrid model integrating uncertainty bonuses and sampling best captures human data in reinforcement learning tasks.