Uncertainty and reward histories have distinct effects on decisions after wins and losses

Intelligent behavior necessitates an adaptive integration of feedback. It is well-known that animals asymmetrically learn from positive and negative feedback. While asymmetrical learning is a robust behavioral effect, the latent computations behind how animals represent their environments and use this to differentially weight wins and losses is poorly understood. Here we tested whether and how uncertainty and reward histories modulate the weights placed on wins and losses using a behavioral data set collected in rats. We propose a reinforcement learning model that integrates uncertainty history via an unsigned average reward prediction error and a separate subjective reward history component. We showed that in a dynamic probabilistic reversal learning task with blocks of variable reward predictability, ongoing estimation of uncertainty history and reward history both distinctly influenced rats' sensitivity to wins and losses. In more predictable environments, and under low uncertainty levels, i.e., when rats were certain in making 'correct' choices, rats weighted wins more than losses, as indicated by a higher win-stay, and lower lose-shift probability. This asymmetrical learning strategy enabled rats to remain with the correct action, while discounting the influence of rare losses. Further, male rats were more impacted by their reward history, i.e., environmental richness, when making lose-shift decisions, but conversely, female rats were more influenced by their uncertainty history. Hence, we found sex-specific contributions of these latent computations in modulating behavior. We overall demonstrate that asymmetrically weighting wins and losses could form an important behavioral strategy when adapting to ongoing changes in reward and uncertainty history.