Learning Generative State Space Models for Active Inference.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Robotics

Background:

• Active inference is a biologically plausible framework for autonomous agents, minimizing prediction error (free energy).
• Existing active inference models often require manually designed generative models, limiting scalability.
• Deep artificial neural networks offer potential for learning complex models from data.

Purpose of the Study:

• To develop a method for learning generative state-space models for active inference using deep learning.
• To enable active inference agents to operate in complex environments without hand-crafted models.
• To demonstrate the scalability and efficiency of active inference with learned models.

Main Methods:

• Utilized deep artificial neural networks to learn generative state-space models from observation-action sequences.
• Applied the learned models to the mountain car problem for validation.
• Tested the approach on high-dimensional pixel observations in simulated (OpenAI Gym car racing) and real-world robotic navigation tasks.

Main Results:

• Learned generative models effectively balanced instrumental value and ambiguity in the mountain car problem.
• Demonstrated successful learning from high-dimensional pixel data in complex environments.
• Active inference policies achieved an order of magnitude greater sample efficiency compared to Deep Q Networks on reinforcement learning tasks.

Conclusions:

• Deep learning enables scalable and autonomous learning of generative models for active inference.
• This approach enhances the applicability of active inference to challenging real-world problems.
• Learned active inference models offer significant sample efficiency advantages over traditional reinforcement learning methods.