Inferring time-varying internal models of agents through dynamic structure learning

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Artificial Intelligence

Background:

• Traditional reinforcement learning models assume fixed agent structures, limiting their ability to model real-world adaptive behaviors and irrationality.
• Understanding how agents dynamically change their learning rules and internal environment representations is crucial for modeling complex decision-making.

Purpose of the Study:

• To introduce a novel dynamic structure learning framework enabling agents to adapt their learning rules and internal representations.
• To investigate the evolution of agent internal structures and learning processes during problem-solving.
• To apply the framework to understand adaptive behaviors in natural intelligence, specifically rat maze navigation.

Main Methods:

• Developed a dynamic structure learning framework to reconstruct the most likely sequence of agent structures based on observed behaviors.
• Utilized a pool of learning rules and environment models for agents to dynamically select from.
• Applied the framework to analyze rat behavior in a maze task, observing changes in maze representation and learning rules.

Main Results:

• Demonstrated that rats progressively refine their maze representation from suboptimal to optimal during learning.
• Observed a transition in learning rules for slower learners from heuristic-based to more rational approaches.
• Showcased the framework's ability to provide insights into the evolution of internal agent models.

Conclusions:

• Dynamic structure learning offers a more realistic approach to modeling agent adaptation and decision-making in complex environments.
• The findings highlight the importance of considering the interplay between learning rules and environment representations for understanding natural intelligence.
• This framework advances the modeling of adaptability, surpassing current artificial intelligence limitations.