[Mathematical models of decision making and learning].

Area of Science:

• Neuroscience
• Computational Psychiatry
• Behavioral Economics

Background:

• Reinforcement learning models are used to analyze neural data in decision-making.
• Selecting appropriate computational models is critical for accurate analysis of neural correlates.
• Standard Q-learning may not fully capture complex choice behaviors.

Purpose of the Study:

• To develop and validate a novel reinforcement learning algorithm for analyzing animal choice behavior.
• To improve the prediction of decision-making processes by incorporating negative reward effects and value forgetting.
• To assess the efficacy of the proposed algorithm against established models.

Main Methods:

• Analysis of choice learning in rats under stochastic reward conditions.
• Development of a generalized reinforcement learning (GRL) algorithm.
• Application of Bayesian estimation for time-varying parameters.
• Comparison of GRL performance against the best Markov model.

Main Results:

• Standard Q-learning inadequately reflects observed choice behaviors in rats.
• The proposed GRL algorithm effectively incorporates negative reward effects and forgetting of unchosen action values.
• GRL demonstrated predictive efficiency comparable to the optimal Markov model for animal choice behaviors.

Conclusions:

• The generalized reinforcement learning (GRL) algorithm provides a more accurate model of choice learning than standard Q-learning.
• GRL's ability to account for reward loss and forgetting enhances its utility for analyzing decision-making.
• The findings support the use of GRL for model-based analysis of neural processes underlying decision-making.