Asymmetric and adaptive reward coding via normalized reinforcement learning.

Area of Science:

• Computational neuroscience
• Cognitive psychology
• Machine learning

Background:

• Reinforcement learning (RL) models learning via prediction errors.
• Standard RL assumes linear reward functions, but neural activity is nonlinear.
• Implications of nonlinear RL for computation and behavior are unclear.

Purpose of the Study:

• Investigate computational and behavioral effects of nonlinear RL.
• Incorporate biologically plausible nonlinear value functions into RL models.
• Explore how nonlinearities impact prediction error coding and learning.

Main Methods:

• Developed a nonlinear RL model incorporating divisive normalization.
• Analyzed the resulting asymmetry in prediction error signals.
• Connected model predictions to behavioral measures of risk preference.
• Evaluated the model's capacity for distributional RL.

Main Results:

• Nonlinear RL with divisive normalization generates tunable prediction error asymmetry.
• This asymmetry explains individual differences in risk preferences.
• The model provides a mechanism for learning reward distributions.
• Demonstrated flexibility in behavioral and computational learning.

Conclusions:

• Biologically valid nonlinear value functions are crucial for RL models.
• Nonlinear RL offers a unified explanation for behavioral and neural findings.
• This approach enhances computational models of learning and decision-making.