Humans adaptively select different computational strategies in different learning environments

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Behavioral Economics

Background:

• The Rescorla-Wagner rule is a dominant model for human reinforcement learning but fails in complex settings.
• Hierarchical extensions exist, but conditions for their adaptive use and human implementation are unclear.

Purpose of the Study:

• To determine when flat versus hierarchical learning strategies are adaptive.
• To investigate which learning strategies humans implement in varying environments.

Main Methods:

• A nested modeling approach was used to evaluate multiple computational models.
• 10 empirical datasets (N=407) across three distinct reinforcement learning environments were analyzed.
• Models were assessed computationally for performance and empirically for human data fit.

Main Results:

• Different environments necessitate different learning strategies for optimal performance.
• Humans adaptively select learning strategies that align with environmental complexity.
• Flat learning models best explained behavior in simple, stable environments.
• Hierarchical learning models were superior in more complex environments.

Conclusions:

• Human learning strategies are flexible and adapt to environmental demands.
• The choice between flat and hierarchical reinforcement learning is driven by environmental complexity.
• Findings support the adaptive selection of learning strategies by humans in reinforcement learning.