Hierarchical Active Inference Using Successor Representations

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Cognitive Science

Background:

• Active inference, based on the free energy principle (FEP), models brain functions like perception and action.
• Scaling active inference to complex, large-scale real-world problems remains a challenge.
• Hierarchical representations are observed in the brain, suggesting their utility in AI.

Purpose of the Study:

• To propose a novel model for hierarchical active inference for action planning.
• To integrate hierarchical environment models with successor representations for efficient planning.
• To demonstrate the benefits of learned hierarchical abstractions in active inference.

Main Methods:

• Developed a hierarchical active inference model combining environment hierarchy and successor representations.
• Investigated how lower-level representations bootstrap higher-level abstract states and actions.
• Applied the model to various planning and reinforcement learning tasks.

Main Results:

• Demonstrated that lower-level successor representations can learn higher-level abstract states.
• Showed that active inference planning can bootstrap learning of abstract actions.
• Confirmed that learned abstractions significantly enhance planning efficiency.

Conclusions:

• This work presents the first application of learned hierarchical state and action abstractions to active inference.
• The proposed hierarchical active inference model effectively addresses challenges in scaling active inference for complex planning.
• The findings support the integration of hierarchical structures in FEP-based theories of brain function.