Computational noise in reward-guided learning drives behavioral variability in volatile environments

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Psychiatry

Background:

• Humans often make suboptimal choices in volatile environments, deviating from maximizing expected value.
• These 'non-greedy' decisions are traditionally viewed as information-seeking behavior or exploration.

Purpose of the Study:

• To investigate the computational basis of non-greedy decisions in volatile environments.
• To determine if behavioral variability stems from exploration or noise in value learning.

Main Methods:

• Utilized reinforcement learning models to simulate behavior.
• Collected multimodal neurophysiological data, including blood oxygen level-dependent (BOLD) responses and pupillary dilation.
• Analyzed trial-to-trial variability in sequential learning steps.

Main Results:

• The majority of non-greedy decisions were attributed to computational noise in action value learning.
• Trial-to-trial learning variability predicted behavior.
• BOLD responses in the dorsal anterior cingulate cortex and pupillary dilation correlated with learning noise.

Conclusions:

• Behavioral variability in reward-guided learning is primarily driven by limited computational precision, not active exploration.
• The locus coeruleus-norepinephrine system may modulate this learning noise.