What is value-accumulated reward or evidence?

Area of Science:

• Cognitive Science
• Neuroscience
• Artificial Intelligence

Background:

• Traditional models explain behavior by maximizing a value function, which can be tautological.
• Optimal control and reinforcement learning rely on pre-defined value functions.
• Understanding the basis of 'value' in behavior is a fundamental question.

Purpose of the Study:

• To propose active inference as an alternative framework for understanding valuable behavior.
• To reframe optimal behavior as a process of Bayesian inference.
• To explain how prior beliefs are optimized hierarchically.

Main Methods:

• Formulating behavior as maximizing log Bayesian evidence.
• Replacing policies with prior beliefs about future states in active inference.
• Utilizing Bayesian inference to specify optimal policies.
• Demonstrating how minimizing uncertainty prescribes prior beliefs.

Main Results:

• Behavior is explained by the accumulation of evidence for internal generative models.
• Optimal behavior is cast as inference: maximizing evidence and minimizing uncertainty.
• Any optimal policy can be represented by prior beliefs within Bayesian inference.
• The imperative to minimize uncertainty dictates the optimization of prior beliefs.

Conclusions:

• Value is equivalent to log Bayesian evidence, driving behavior through model inference.
• Active inference offers a non-tautological explanation for goal-directed behavior.
• This framework provides a unified account of policy selection and belief optimization.