Individual differences in tail risk sensitive exploration using Bayes-adaptive Markov decision processes.

Area of Science:

• Behavioral Ecology
• Computational Neuroscience
• Reinforcement Learning

Background:

• Exploration in novel environments presents a trade-off between resource discovery and predation risk.
• Individual differences in risk sensitivity and prior expectations influence animal behavior.
• Existing models struggle to capture the nuances of risk-sensitive exploration.

Purpose of the Study:

• To develop a computational model explaining individual differences in risk-sensitive exploration behavior.
• To integrate risk aversion and threat perception into an exploration framework.
• To link behavioral patterns to underlying psychological traits.

Main Methods:

• Constructed a Bayes-adaptive Markov decision process model.
• Incorporated an adaptive hazard function for predation risk.
• Included an intrinsic reward function for exploration and a Conditional Value at Risk (CVaR) objective for risk sensitivity.
• Fitted the model to behavioral data from 26 animals exploring a novel object.

Main Results:

• The model accurately captured quantitative (bout frequency/duration) and qualitative (approach styles) aspects of exploration.
• Individual differences in exploration strategies were explained by model parameters.
• Identified distinct behavioral profiles: risk-neutral (flexible priors) and risk-averse (inflexible priors).

Conclusions:

• Bayes-adaptive modeling provides a framework for understanding risk-sensitive exploration.
• Individual differences in exploration are linked to distinct risk-aversion and threat-prior profiles.
• This approach may inform the study and treatment of psychiatric disorders like anxiety.