Active Inference: Demystified and Compared.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Autonomous agents operate in dynamic environments, a challenge addressed by both active inference and reinforcement learning.
• Limited comparative studies exist between active inference and reinforcement learning in discrete-state environments.

Purpose of the Study:

• Provide an accessible overview of discrete-state active inference.
• Compare active inference and reinforcement learning behaviors in a discrete-state OpenAI gym environment.
• Highlight natural behaviors in active inference often engineered in reinforcement learning.

Main Methods:

• Condensed overview of active inference literature through a reinforcement learning lens.
• Discrete-state comparison using an OpenAI gym baseline.
• Analysis of agent behaviors in reward-free scenarios using Q-learning and Bayesian model-based reinforcement learning.

Main Results:

• Active inference agents perform Bayes-optimal epistemic exploration and handle environmental uncertainty.
• Active inference removes reliance on explicit reward signals, treating rewards as preferred observations.
• Agents learn behaviors through preference learning even in reward-free settings.

Conclusions:

• Active inference offers a unified framework for understanding agent behavior, including exploration and preference learning.
• The discrete-state formulation of active inference can be applied to complex tasks like robotics and games.
• This work demystifies active inference by demonstrating its behaviors alongside reinforcement learning agents in a benchmark environment.