Learning and forgetting using reinforced Bayesian change detection.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Behavioral Economics

Background:

• Agents in volatile environments need to detect changes without overreacting to noise.
• Adaptive learning rates are crucial in reinforcement learning (RL) for reliable model updates.
• Animal behavior shows reduced flexibility after training in stable environments, suggesting adaptive learning.

Purpose of the Study:

• To develop a novel Bayesian learning model for adaptive change detection in RL.
• To enable dynamic adjustment of learning rates based on environmental volatility.
• To implement a critic in an actor-critic RL model that optimizes decision-making.

Main Methods:

• Utilized variational inference for a new Bayesian learning model.
• Introduced Stabilized Forgetting to update beliefs using a mixture of priors and posteriors.
• Optimized model parameters, including the weight of prior/posterior beliefs, dynamically.

Main Results:

• The model successfully emulates various adaptation strategies based on prior environmental stability assumptions.
• Model parameters demonstrated high accuracy when fit to real-world data.
• The model exhibits trade-offs between flexibility and computational cost, mirroring empirical observations.

Conclusions:

• The proposed Bayesian model offers a general framework for studying learning flexibility and decision-making in RL.
• It provides a mechanism for agents to adapt to changing environmental contingencies effectively.
• The model's ability to balance adaptation and stability offers insights into biological learning systems.