Fuzzy Knowledge-Based Hierarchical Reinforcement Learning for Large-Scale Heterogeneous Multiagent Systems

Multiagent reinforcement learning (MARL) has garnered extensive research attention due to its strong learning capabilities, leading to its deployment in increasingly challenging scenarios. Although progress has been made toward more generalizable solutions, many MARL algorithms continue to struggle with balancing scalability and heterogeneity, particularly under conditions of growing uncertainty. Research has shown that combining dense local interactions with sparse global interactions can significantly enhance scalability while preserving agent heterogeneity. Motivated by these insights and inspired by human social behavior, we propose a novel hierarchical method that integrates human guidance with multiagent systems (MASs). Rather than requiring agents to learn from scratch, our method transfers abstract knowledge from humans, employing fuzzy logic to manage the inherent uncertainty in this guidance and reduce the required human effort. To accommodate both local and global interactions, we introduce two levels of human guidance: individual action guidance for agents and an attention graph to describe agent relationships. Our proposed approach is end-to-end and compatible with diverse MARL algorithms. We evaluate our approach in the starcraft multiagent challenge (SMAC) and SMACv2 environments. Empirical results demonstrate its effectiveness, even under low-performance fuzzy human guidance.