Learning action-oriented models through active inference.

Area of Science:

• Computational neuroscience
• Machine learning
• Behavioral ecology

Background:

• Organisms learn probabilistic environmental models, but practical learning methods are unclear.
• Action-oriented models offer parsimonious representations but can lead to sub-optimal solutions (bad-bootstraps) when learning is solely goal-directed.
• Natural environments' complexity makes comprehensive modeling infeasible.

Purpose of the Study:

• To demonstrate how active inference can enable efficient learning of action-oriented models.
• To investigate the trade-offs between goal-directed and epistemic learning strategies.
• To provide a principled approach for sample-efficient learning in adaptive agents.

Main Methods:

• Utilized the normative framework of active inference.
• Developed a simple agent-based model of bacterial chemotaxis.
• Compared learning outcomes from goal-directed, epistemic, and combined active inference strategies.

Main Results:

• Goal-directed learning alone leads to behaviorally relevant but sub-optimal models.
• Epistemic learning yields comprehensive but inefficient models.
• Active inference agents learn parsimonious, action-tailored models, avoiding bad-bootstraps and sub-optimal convergence.
• Models learned via active inference support adaptive behavior despite non-veridical representations.

Conclusions:

• Balancing goal-directed and epistemic behaviors via active inference is crucial for efficient model learning.
• Active inference offers a principled method for learning adaptive models from limited environmental interactions.
• This approach provides a pathway towards sample-efficient learning algorithms for artificial and biological agents.