An algorithmic account for how humans efficiently learn, transfer, and compose hierarchically structured decision...

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Artificial Intelligence

Background:

• Human intelligence exhibits remarkable flexibility through hierarchical decision policies.
• Understanding the computational mechanisms of learning and constructing these policies is crucial.

Purpose of the Study:

• To investigate the learning processes underlying human hierarchical policy formation.
• To develop a computational model explaining observed human decision-making strategies.

Main Methods:

• Conducted a large-scale decision-making experiment with 1026 participants making over 1 million choices.
• Developed a novel algorithmic account integrating reinforcement learning, policy compression, meta-learning, and Bayesian inference.

Main Results:

• Human participants demonstrated the ability to learn, transfer, and recompose hierarchical policies.
• Behavioral data supported a model where initial compressed policies gradually unfold into hierarchical structures.
• Algorithmic modeling indicated a temporally backward structure for learned hierarchical policies.

Conclusions:

• Human decision-making relies on a dynamic interplay of reinforcement learning, policy compression, meta-learning, and working memory.
• This process supports resource-rational, compositional decision-making and policy abstraction.
• The findings offer insights into the computational underpinnings of flexible human intelligence.