Information uncertainty influences learning strategy from sequentially delayed rewards.

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Behavioral Economics

Background:

• Temporal credit assignment, determining which past event caused a reward, is challenging due to environmental uncertainty.
• Existing research often isolates delay and reward dimensions, leaving algorithmic solutions and uncertainty effects underexplored.

Purpose of the Study:

• To investigate human strategies for temporal credit assignment under varying levels of information uncertainty.
• To compare the efficacy of two computational models: eligibility trace and tabular update.

Main Methods:

• Adapted a reward learning task with delayed rewards, intervening events, and manipulated information uncertainty.
• Developed and compared two computational learning models: eligibility trace and tabular update.
• Validated model predictions against human participant behavior (N=142).

Main Results:

• Both models learned the task and predicted participant choices and credit assignment signatures.
• The tabular model significantly outperformed the eligibility trace model under low information uncertainty.
• Reduced uncertainty correlated with increased use of the tabular strategy and higher tabular weights.

Conclusions:

• Human temporal credit assignment strategy adapts based on environmental information uncertainty.
• The tabular update strategy is more effective than the eligibility trace when uncertainty is low.
• Findings offer insights into adaptive learning mechanisms in dynamic environments.