Free energy, value, and attractors.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Statistical Physics

Background:

• Action and perception are increasingly understood through the lens of minimizing free energy in sensory data.
• Current free energy accounts do not incorporate reward or cost functions central to reinforcement learning and optimal control theory.

Purpose of the Study:

• To investigate whether reward is a necessary component for explaining adaptive behavior.
• To bridge the conceptual gap between free energy minimization and reinforcement learning frameworks.

Main Methods:

• The study connects the free energy formulation, which explains action via minimizing sensory surprise, with reinforcement learning, which optimizes policies for reward maximization.
• It analyzes the mathematical correspondence between these two approaches.

Main Results:

• Optimal policies in this integrated framework correspond to empirical priors over hidden environmental states.
• These priors guide agents to actively seek states that are both expected and valuable.

Conclusions:

• Adaptive behavior can be explained by integrating free energy minimization with reward-based learning.
• Agents are driven to explore and attain states that align with their predictive models and expected value.