Sophisticated Learning: A novel algorithm for active learning during model-based planning.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Reinforcement Learning

Background:

• Active Inference (AI) models decision-making under uncertainty.
• Sophisticated Inference (SI) improves AI for multi-step planning.
• Limited comparison exists between SI and established Reinforcement Learning (RL) algorithms.

Purpose of the Study:

• Compare SI performance against Bayesian RL schemes.
• Introduce and evaluate Sophisticated Learning (SL), an extension of SI.
• SL integrates active learning into planning by considering future observational learning.

Main Methods:

• Developed a novel, biologically inspired environment.
• Environment necessitates balancing goal-seeking with active information acquisition.
• Simulated and compared SI, SL, Bayes-adaptive RL, and Upper Confidence Bound (UCB) algorithms.

Main Results:

• Sophisticated Learning (SL) demonstrated superior performance across all tested algorithms.
• SL outperformed Bayes-adaptive RL and UCB, which use similar directed exploration principles.
• The novel environment highlighted SL's unique capability in balancing exploration and exploitation.

Conclusions:

• Active Inference, particularly with SL, is effective for complex, biologically relevant planning problems.
• SL provides a novel approach to counterfactual reasoning and active learning in planning agents.
• Findings support AI's utility and offer tools for cognitive science research.