Abstract inference of unchosen option values

Area of Science:

• Cognitive Neuroscience
• Reinforcement Learning
• Decision Making

Background:

• Reinforcement learning (RL) investigates how reward feedback influences behavior, questioning if it leads to simple associations or complex inferences.
• Reversal learning tasks suggest humans can infer value changes, but the underlying cognitive mechanisms remain unclear.

Purpose of the Study:

• To investigate whether humans utilize abstract rules to infer the values of unchosen alternatives in RL tasks.
• To differentiate between stimulus-specific context learning and abstract contingency structure deduction.

Main Methods:

• Two experiments were conducted, probing human inference of unchosen stimulus values using modified discrimination problems.
• Experiment 1 involved 37 participants in a task adapted from monkey studies, manipulating stimulus replacement.
• Experiment 2 used fMRI-compatible tasks with 17 participants to examine neural correlates and behavioral differences.

Main Results:

• Participants in the abstract inference group learned to make accurate choices but committed significantly more errors than controls.
• This suggests that inferences about unchosen stimulus values are infrequently employed in standard discrimination tasks.
• Experiment 2 corroborated these findings and revealed substantial individual differences in explicit awareness of novel stimulus values.

Conclusions:

• Humans can employ abstract rules for unchosen value inference in RL, but this strategy is not always dominant or efficient.
• The findings highlight the complexity of cognitive operations in RL and suggest limitations in applying abstract reasoning to infer unchosen options.
• Individual differences in awareness underscore the varied cognitive strategies employed in decision-making under uncertainty.