Active inference and agency: optimal control without cost functions.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Decision Theory

Background:

• Decision-making under uncertainty is often modeled using Markov decision problems (MDPs).
• Optimal control seeks to minimize costs or maximize rewards over time.
• Partially observable environments pose challenges due to hidden states.

Purpose of the Study:

• To formulate optimal control within a variational free-energy framework.
• To demonstrate that optimal control is equivalent to active inference.
• To introduce a unified approach for decision-making under uncertainty.

Main Methods:

• Variational free-energy formulation applied to partially observable MDPs.
• Casting optimal control as active inference, minimizing a free energy bound.
• Integrating reward/cost functions into prior beliefs of a generative model.
• Employing Bayesian filtering techniques for inference.

Main Results:

• Optimal control is shown to be a form of active inference.
• Reward and cost functions are naturally incorporated into prior beliefs.
• Optimal control is reframed as a pure inference problem.
• A distinction is made between agency-free and agency-based generative models.

Conclusions:

• Active inference provides a unified framework for optimal control and decision-making.
• The formulation allows for the application of standard Bayesian inference techniques.
• Modeling control as a hidden state introduces agency into generative models.