Generating adaptive behaviour within a memory-prediction framework.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• The Memory-Prediction Framework (MPF) and Hierarchical-Temporal Memory (HTM) are established for unsupervised learning (classification, prediction).
• Previous applications have not explored MPF/HTM for reinforcement learning or adaptive control systems.
• The human neocortex is crucial for behavior generation, and MPF models the neocortex.

Purpose of the Study:

• To incorporate MPF/HTM into reinforcement learning for agent behavior control.
• To develop a biologically-plausible framework for adaptive systems that generate behavior.
• To explore and interact with an external world while maximizing a reward function.

Main Methods:

• Proposed simple, biologically-plausible enhancements to the MPF.
• Enabled the MPF hierarchy to explore and interact with an external environment.
• Utilized reinforcement learning for hierarchy development from a random initial state.

Main Results:

• Demonstrated that behavior generation and control occur entirely within the MPF hierarchy.
• Showcased successful learning and interaction through reinforcement learning and environmental exploration.
• A 2-node hierarchy learned to play 'rocks, paper, scissors' against a predictable opponent.

Conclusions:

• The enhanced MPF can be effectively used for reinforcement learning and adaptive control.
• This framework provides a biologically-plausible model for agent behavior generation.
• The study validates the potential of MPF/HTM in creating intelligent, adaptive systems.