A bayesian foundation for individual learning under uncertainty.

Area of Science:

• Computational neuroscience
• Cognitive science
• Machine learning

Background:

• Current computational learning models like reinforcement learning (RL) and Bayesian learning have limitations.
• Bayesian models often struggle with inter-individual variability and complex calculations for online learning.

Purpose of the Study:

• To introduce a generic hierarchical Bayesian framework for individual learning.
• To model learning under multiple forms of uncertainty, including environmental volatility and perceptual uncertainty.
• To derive efficient, trial-by-trial update equations interpretable within RL.

Main Methods:

• Developed a hierarchical Bayesian framework with Gaussian random walks across levels.
• Employed variational Bayes with a mean-field approximation and a novel posterior energy function approximation.
• Derived analytical, efficient trial-by-trial update equations.

Main Results:

• The model enables real-time, analytical learning updates.
• It provides a natural interpretation in terms of reinforcement learning.
• Includes parameters for individual differences in learning, such as precision-weighting of prediction error.

Conclusions:

• The framework offers a novel foundation for understanding normal and pathological learning.
• It contextualizes reinforcement learning within a broader Bayesian scheme.
• The model's generality allows for diverse applications in discrete/continuous states and varying uncertainty levels.