Gradient-Free De Novo Learning.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Learning goal-directed behavior from scratch (de novo learning) is a key challenge in artificial intelligence and neuroscience.
• Current methods often require pre-defined models or extensive training data.
• Understanding how agents discover optimal policies without prior knowledge is crucial for developing more autonomous systems.

Purpose of the Study:

• To apply active inference to the problem of de novo learning for sequential policy optimization.
• To develop a procedure that discovers the structure and parameters of a discrete generative model directly from observations.
• To demonstrate how the free energy principle can reframe learning as the discovery of attracting sets in generative model dynamics.

Main Methods:

• Active inference framework applied to de novo learning.
• A procedure that grows and reduces a generative model to find a pullback attractor over generalized states.
• Reframing the learning problem using the free energy principle.
• Comparison with value-based formulations like Bellman optimality.

Main Results:

• The proposed method successfully learns a generative model that features an attracting set, representing paths of least action.
• This attracting set guides the agent towards goal states while avoiding costly states.
• Demonstrated de novo structure learning and emergent agency in a simulated arcade game.

Conclusions:

• Active inference provides an efficient framework for de novo learning of goal-directed behavior.
• Learning can be framed as discovering attracting sets within generative models under the free energy principle.
• The approach facilitates the discovery of policies and agentic behavior from observational data.