An inductive bias for slowly changing features in human reinforcement learning.

Area of Science:

• Cognitive Science
• Neuroscience
• Machine Learning

Background:

• Efficient behavior requires identifying goal-relevant features in new environments.
• Prior knowledge about reward-predicting features, like their rate of change, may guide this process.
• Behaviorally relevant processes often change slower than random noise.

Purpose of the Study:

• To investigate if humans exhibit a bias to learn more effectively when task-relevant features change slowly versus quickly.
• To determine if humans utilize prior knowledge about feature dynamics in reinforcement learning.

Main Methods:

• 295 human participants completed two experiments and a replication task involving learning rewards from two-dimensional bandits.
• Participants learned rewards associated with either slowly or quickly changing features of the bandits.
• Computational modeling using Kalman filters analyzed learning rates and adjustments based on feature relevance and speed.

Main Results:

• Participants accrued more reward when the relevant feature changed slowly and the irrelevant feature changed quickly.
• No significant difference was found in generalization to unseen feature values across conditions.
• Human learning rates were higher for slow features, and participants adjusted learning based on feature relevance and speed.

Conclusions:

• Human reinforcement learning demonstrates a bias favoring slower-changing features for reward prediction.
• This bias suggests a specific strategy in how humans approach learning in dynamic environments.
• Learning rate adjustments are sensitive to both the relevance and temporal dynamics of environmental features.