Active Inference and Reinforcement Learning: A Unified Inference on Continuous State and Action Spaces Under Partial...

Area of Science:

• Artificial Intelligence
• Computational Neuroscience

Background:

• Reinforcement learning (RL) excels in fully observable environments but struggles with partial observations common in real-world scenarios.
• Partially Observable Markov Decision Processes (POMDPs) model these complex environments, but traditional RL methods face challenges with long time horizons and high-dimensional data.
• Active Inference (AIF) offers an alternative by minimizing expected free energy (EFE), balancing exploration and exploitation, yet is limited by computational demands in large spaces.

Purpose of the Study:

• To propose a unified principle connecting RL and AIF for enhanced agent decision-making in POMDPs.
• To overcome the limitations of existing RL and AIF methods in continuous, high-dimensional, and long-horizon partially observable environments.
• To demonstrate a novel approach that integrates reward-maximizing and information-seeking behaviors.

Main Methods:

• Developed a theoretical framework unifying RL and AIF principles.
• Formulated a novel approach applicable to continuous space POMDPs.
• Conducted rigorous theoretical analysis and experimental validation.

Main Results:

• Established a theoretical connection between AIF and RL, enabling seamless integration.
• Demonstrated superior learning capabilities in continuous space POMDPs compared to existing RL methods.
• Showcased the ability to solve reward-free problems by leveraging information-seeking exploration.

Conclusions:

• The unified principle effectively bridges RL and AIF, overcoming limitations of individual approaches.
• The proposed method offers a powerful new tool for designing artificial agents capable of handling complex, partially observable environments.
• This work opens new avenues for AIF in artificial intelligence, reducing reliance on explicit reward specification.