Exploring Feature Dimensions to Learn a New Policy in an Uninformed Reinforcement Learning Task

Area of Science:

• Cognitive Neuroscience
• Decision Making
• Reinforcement Learning

Background:

• Limited information necessitates exploration through trial-and-error to understand relationships.
• Human exploratory behavior under information scarcity is understudied.
• Understanding how and why humans explore new feature dimensions is crucial for learning new policies.

Purpose of the Study:

• Investigate human exploratory behavior in learning new state-space policies with limited information.
• Examine both behavioral and neural mechanisms of feature exploration.
• Determine if values are transferred or relearned during exploration.

Main Methods:

• Designed a novel multi-dimensional reinforcement learning task.
• Utilized a reinforcement learning algorithm to model policy exploration and learning.
• Analyzed behavioral and neural data to understand exploration processes.

Main Results:

• Provided first evidence of value transfer from previous to new online policies during feature exploration.
• Demonstrated that exploration is regulated by cognitive ambiguity.
• Identified potential neural control of exploration by the frontopolar cortex.

Conclusions:

• Human exploration of new feature dimensions involves value transfer, not relearning from scratch.
• Cognitive ambiguity plays a regulatory role in exploration.
• The frontopolar cortex may be involved in controlling this exploratory process.
• Findings offer new insights into understanding feature exploration in open-ended environments with limited information.