Adaptive learning via selectionism and Bayesianism, Part II: the sequential case.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Animals adjust action tendencies based on past reinforcement (positive or negative).
• Selectionist learning dynamics resemble Bayesian learning in how rewards influence action probabilities.
• The temporal credit-assignment problem is crucial for learning action sequences.

Purpose of the Study:

• To explore the equivalence between selectionist and Bayesian learning in solving the temporal credit-assignment problem.
• To investigate the emergence of secondary reinforcement and actor-critic architectures.
• To examine the impact of concurrent action learning and reward prediction in on-line schemes.

Main Methods:

• Analysis of selectionist learning dynamics.
• Exploration of Bayesian learning principles applied to action sequences.
• Investigation of actor-critic architectures for sequential decision-making.

Main Results:

• Demonstrated equivalence between selectionist and Bayesian learning for action sequence optimization.
• Identified secondary reinforcement as a predictor of average stimulus-associated reward.
• Showcased actor-critic architectures for concurrent action and reward learning.

Conclusions:

• Adaptive learning in animals shares principles with Bayesian inference and reinforcement learning.
• Actor-critic models provide a framework for understanding sequential decision-making and reward prediction.
• Concurrent on-line learning of actions and rewards is feasible and effective.