Novelty and Inductive Generalization in Human Reinforcement Learning.

Area of Science:

• Cognitive Science
• Neuroscience
• Machine Learning

Background:

• Reinforcement learning (RL) decision-makers face uncertainty regarding novel options.
• Generalizing past experiences to new choices is a key challenge in RL.

Purpose of the Study:

• To investigate how hierarchical Bayesian inference can address the problem of predicting the value of untried options in RL.
• To explore the equivalence between Bayesian models and temporal difference learning algorithms in modeling human and animal RL.
• To test the hypothesis that humans utilize structured inductive knowledge for efficient search and prediction of novel options.

Main Methods:

• Developed a hierarchical Bayesian inference model to generalize experience to novel options.
• Established an equivalence between the proposed Bayesian model and temporal difference learning algorithms.
• Conducted two behavioral experiments to test model predictions regarding human learning and prediction of novel options.

Main Results:

• Demonstrated that hierarchical Bayesian inference effectively models the prediction of novel option values.
• Provided evidence for an equivalence between Bayesian approaches and temporal difference learning in RL.
• Behavioral experiments confirmed that humans leverage structured inductive knowledge to predict novel options, enhancing search efficiency.

Conclusions:

• Hierarchical Bayesian inference offers a powerful framework for understanding RL, particularly in generalizing knowledge to novel situations.
• The model provides a new perspective on human and animal decision-making, highlighting the role of abstract knowledge.
• The findings suggest a novel interpretation of dopaminergic responses to novelty within the context of RL and inductive reasoning.