The algorithmic architecture of exploration in the human brain

Area of Science:

• Neuroscience
• Cognitive Science
• Machine Learning

Background:

• Exploration versus exploitation is a fundamental challenge in reinforcement learning.
• Recent research highlights diverse algorithmic strategies humans employ for exploration.
• These strategies involve distinct neural systems and developmental paths.

Purpose of the Study:

• To review recent studies on exploration strategies in reinforcement learning.
• To examine the role of structural knowledge in human exploration.
• To introduce computational models for understanding the neural basis of reinforcement learning.

Main Methods:

• Literature review of recent studies on reinforcement learning and exploration.
• Analysis of human exploration strategies, including random and uncertainty-directed approaches.
• Discussion of computational models inspired by machine learning.

Main Results:

• Humans utilize a combination of random and uncertainty-directed exploration.
• Sophisticated structural knowledge, like option correlations, aids human exploration.
• New computational models are emerging to formalize these concepts.

Conclusions:

• Understanding human exploration requires considering multiple strategies and knowledge integration.
• Computational models provide a framework for investigating the neural underpinnings of reinforcement learning.
• Future research can leverage these models to further elucidate brain mechanisms.