Separating random and deterministic sources of computational noise in explore-exploit decisions.

Area of Science:

• Cognitive Science
• Neuroscience
• Machine Learning

Background:

• Human decision-making exhibits inherent variability, often perceived as suboptimal.
• Theoretical and empirical work suggests this variability can be adaptive, particularly in explore-exploit scenarios.
• Randomness in exploration encourages considering novel options, a behavior observed when exploration value increases.

Purpose of the Study:

• To investigate whether behavioral variability in 'random exploration' is truly random or driven by unobserved deterministic processes.
• To quantify the proportion of variability attributable to stimulus-driven deterministic processing versus random processes.
• To explore the relationship between deterministic and random variability sources as exploration value changes.

Main Methods:

• Designed an explore-exploit task where participants faced identical choices twice, unbeknownst to them.
• Modeled participant behavior to estimate bounds on deterministic (stimulus-driven) and random variability.
• Analyzed how these variability components changed with the increasing value of exploration.

Main Results:

• At least 14% of the variability in random exploration was accounted for by deterministic stimulus processing.
• This suggests that up to 86% of the variability could be truly random, though potentially influenced by other deterministic factors.
• Both deterministic and random variability sources scaled proportionally with increasing exploration value.

Conclusions:

• Human exploration variability is not entirely random; a significant portion is deterministically linked to stimulus processing.
• A common noise gating mechanism may regulate both deterministic and random components of exploration variability.
• Understanding the sources of variability is crucial for modeling adaptive decision-making processes.