Inferring Latent Temporal Sparse Coordination Graph for Multiagent Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Effective coordination is vital for cooperative multiagent reinforcement learning (MARL).
• Existing graph learning methods in MARL are limited by their reliance on single-step observations, leading to inefficient information exchange.
• High computational costs in dense graphs hinder the scalability of current MARL approaches.

Purpose of the Study:

• To propose a novel method for inferring a latent temporal sparse coordination graph (LTS-CG) for MARL.
• To address limitations of existing methods by incorporating historical data and reducing computational complexity.
• To enhance agent collaboration and knowledge exchange through dynamic graph structures.

Main Methods:

• Leveraging historical observations to compute an agent-pair probability matrix for sparse graph sampling.
• Implementing Predict-Future and Infer-Present mechanisms to capture temporal dependencies and environmental context.
• Employing an end-to-end approach for simultaneous graph learning and agent training.

Main Results:

• The proposed LTS-CG method demonstrates superior performance on the StarCraft II benchmark.
• The approach effectively captures agent dependencies and relationship uncertainty.
• Computational complexity is reduced, scaling linearly with the number of agents.

Conclusions:

• LTS-CG offers an effective solution for agent coordination in MARL by utilizing temporal information and sparse graphs.
• The method enhances collaboration and knowledge exchange, leading to improved learning and performance.
• LTS-CG provides a scalable and computationally efficient approach for complex MARL tasks.