Free Energy Projective Simulation (FEPS): Active inference with interpretability

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Complex Systems

Background:

• The Free Energy Principle (FEP) and Active Inference (AIF) offer a unified framework for understanding learning, cognition, perception, and action in self-organizing systems.
• Reinforcement Learning (RL) agents, often employing deep neural networks, have been developed to perform active inference tasks, with recent efforts focusing on enhancing performance in complex environments.

Purpose of the Study:

• To develop an interpretable agent modeling approach within the constraints of FEP and AIF, avoiding deep neural networks.
• To introduce Free Energy Projective Simulation (FEPS) as a novel method for agent modeling and policy optimization.

Main Methods:

• FEPS agents utilize internal rewards to build world models of partially observable environments.
• Policies are derived by minimizing expected free energy, a core tenet of AIF.
• Techniques for managing long-term goals and mitigating prediction errors from hidden state estimation are incorporated.

Main Results:

• FEPS agents successfully resolved ambiguity in two RL environments (timed response and partially observable navigation) inspired by behavioral biology.
• Agents demonstrated the ability to contextualize observations based solely on prediction accuracy.
• Optimal policies were inferred flexibly for diverse target observations within the environments.

Conclusions:

• FEPS provides an interpretable alternative to deep neural networks for modeling agents under FEP and AIF.
• The model effectively handles environmental ambiguity and optimizes policies through prediction accuracy-based contextualization.
• FEPS shows promise for developing adaptive and goal-directed agents in complex, partially observable settings.