Framework for hierarchical deep reinforcement learning with conceptual embedding.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep reinforcement learning (DRL) struggles with large, combinatorial state-action spaces.
• Hierarchical DRL (HDRL) offers a solution for scalability but lacks efficient structure design.
• Current HDRL methods face challenges in creating effective hierarchical policies.

Purpose of the Study:

• To propose a general HDRL framework utilizing conceptual embedding to constrain the exploration space.
• To formalize recognition-decision decoupling within a hierarchical policy structure for the first time.
• To clarify the relationship between abstract state and goal spaces for transparent inference.

Main Methods:

• Developed a novel HDRL framework incorporating conceptual embedding.
• Implemented recognition-decision decoupling within the hierarchical policy.
• Defined and analyzed the complexity of the exploration space under the proposed framework.

Main Results:

• The framework successfully restricts the exploration space through conceptual embedding.
• Demonstrated a transparent inference pipeline enabling structured reasoning and prior knowledge integration.
• Experimental validation confirmed the framework's effectiveness in improving exploration efficiency.

Conclusions:

• The proposed HDRL framework with conceptual embedding addresses scalability challenges in DRL.
• This approach facilitates efficient policy learning and exploration by leveraging abstract concepts.
• The method offers a structured and transparent approach to complex decision-making in AI.