Apparent learning biases emerge from optimal inference: Insights from master equation analysis

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Reinforcement Learning

Background:

• Previous research suggests human decision-making in Bernoulli bandit tasks exhibits positivity and confirmation biases.
• These biases are often inferred by fitting Q-learning models with constant learning rates to human data.
• This implies humans may not objectively integrate new information.

Purpose of the Study:

• To investigate whether apparent biases in human decision-making arise from objective Bayesian inference or actual cognitive biases.
• To analyze the dynamics of learning systems using Master equations to understand behavioral signatures.
• To determine if temporally varying learning rates are necessary for accurate modeling of human behavior.

Main Methods:

• Applied Bayesian inference to model belief updating in a two-armed Bernoulli bandit task.
• Analyzed the stochastic dynamics of both Bayesian inference and Q-learning models using Master equations.
• Compared behavioral signatures, specifically action switching probabilities, across different learning models.

Main Results:

• Even objective Bayesian inference, when approximated by a Q-learning model, can appear to exhibit positivity and confirmation biases.
• Bayesian inference, when formulated as an effective Q-learning algorithm, has unbiased, temporally decreasing learning rates.
• Both confirmation bias and temporally decreasing learning rates share a behavioral signature of decreased action switching probabilities.

Conclusions:

• Apparent biases in human decision-making may be artifacts of simplified model fitting, not necessarily inherent cognitive biases.
• Temporally decreasing learning rates, consistent with objective Bayesian inference, can mimic confirmation bias.
• Accurate modeling of human behavior requires considering temporally varying learning rates before concluding biased decision-making.