Fuzzy Knowledge-Based Hierarchical Reinforcement Learning for Large-Scale Heterogeneous Multiagent Systems.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multiagent reinforcement learning (MARL) faces challenges in balancing scalability and heterogeneity, especially with increasing uncertainty.
• Combining dense local and sparse global interactions can improve MARL scalability and preserve agent heterogeneity.
• Human social behavior offers insights for designing more effective MARL systems.

Purpose of the Study:

• To propose a novel hierarchical method integrating human guidance into multiagent systems (MASs).
• To leverage abstract knowledge transfer from humans, using fuzzy logic to manage uncertainty and reduce human effort.
• To enhance MARL scalability and heterogeneity through a structured human guidance approach.

Main Methods:

• Developed a hierarchical method combining individual action guidance and an attention graph for agent relationships.
• Employed fuzzy logic to manage uncertainty in human guidance, enabling abstract knowledge transfer.
• Designed an end-to-end approach compatible with various MARL algorithms.

Main Results:

• Empirical evaluation in StarCraft Multi-Agent Challenge (SMAC) and SMACv2 environments demonstrated the approach's effectiveness.
• The method showed significant improvements in scalability and heterogeneity.
• The approach proved effective even with low-performance, fuzzy human guidance.

Conclusions:

• The proposed hierarchical method successfully integrates human guidance into MARL systems.
• Fuzzy logic effectively manages uncertainty in human guidance, enhancing MARL performance.
• This approach offers a promising direction for developing more robust and scalable MARL solutions.