Feudal Latent Space Exploration for Coordinated Multi-Agent Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multi-agent reinforcement learning (MARL) faces challenges with independent exploration as agent numbers increase, leading to exponential complexity.
• Efficient coordination is crucial for multi-agent systems to achieve optimal performance in complex tasks.

Purpose of the Study:

• To propose a novel method, feudal latent-space exploration (FLE), to address the challenges of coordinated exploration in MARL.
• To enhance the efficiency and coordination capabilities of multi-agent reinforcement learning systems.

Main Methods:

• Introduced a feudal commander to learn a low-dimensional global latent structure for guiding agent exploration.
• Employed multi-agent policy gradient (PG) for end-to-end optimization of agent policies and the latent structure.
• Validated the approach in two multi-agent environments requiring explicit coordination.

Main Results:

• FLE demonstrated superior performance compared to baseline MARL approaches with independent exploration.
• The method showed improvements in mean rewards, exploration efficiency, and the expressiveness of coordination policies.
• Experimental results confirmed the effectiveness of the proposed feudal exploration framework.

Conclusions:

• Feudal latent-space exploration (FLE) offers an effective solution for coordinated exploration in multi-agent reinforcement learning.
• The proposed framework significantly enhances learning efficiency and coordination capabilities in MARL systems.
• FLE provides a scalable approach to tackling complex coordination problems with an increasing number of agents.