Hierarchical memory-based deep reinforcement learning in simulated survival environments.

Area of Science:

• Artificial Intelligence
• Neuroscience
• Cognitive Science

Background:

• Deep reinforcement learning (DRL) agents struggle with memory interference, exploration in sparse reward settings, and adaptability in dynamic environments.
• These limitations hinder performance in complex, long-horizon decision-making tasks.
• Existing DRL methods lack sophisticated memory systems to address these challenges.

Purpose of the Study:

• To propose a novel hierarchical memory-based deep reinforcement learning (HM-DRL) architecture inspired by neuroscience.
• To enhance agent performance in complex decision-making tasks, particularly those with long horizons and dynamic changes.
• To improve learning efficiency and adaptability in sparse reward scenarios.

Main Methods:

• Developed a HM-DRL architecture integrating three memory layers: perceptual, episodic, and abstract.
• Implemented a dynamic gating mechanism for policy network and memory integration.
• Incorporated a compound reward function to optimize policy and crisis response.
• Utilized an open-ended, multi-task simulated survival environment for testing.

Main Results:

• HM-DRL significantly mitigated memory interference in long-horizon tasks.
• The architecture demonstrated enhanced adaptability to sudden crises and dynamic environmental changes.
• Learning efficiency was improved, especially under sparse reward conditions.
• The causal graph construction in the abstract memory layer supported backward reasoning.

Conclusions:

• The proposed HM-DRL architecture offers an effective solution for memory management challenges in long-horizon decision-making.
• This approach lays the groundwork for developing agents with advanced causal reasoning capabilities.
• HM-DRL represents a significant advancement in creating more robust and adaptable AI agents.