Dynamic Deep Factor Graph for Multi-Agent Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multi-agent reinforcement learning (MARL) faces challenges in coordinating agents effectively.
• Global value functions suffer from the curse of dimensionality, while fully decomposed methods can overgeneralize.
• Existing coordination methods struggle with dynamic collaboration patterns.

Purpose of the Study:

• To introduce Dynamic Deep Factor Graphs (DDFG) for improved value decomposition in MARL.
• To enable adaptive coordination by learning dynamic graph structures.
• To provide theoretical insights into the trade-offs of high-order decompositions.

Main Methods:

• DDFG represents the global value using factor graphs and learns graph structures dynamically.
• A graph-generation policy adapts to evolving inter-agent relationships.
• Max-sum inference is employed for efficient joint policy derivation.

Main Results:

• DDFG demonstrates consistent performance gains over strong baselines in predator-prey and SMAC tasks.
• The method shows improved sample efficiency and robustness in complex MARL settings.
• Theoretical analysis provides guidance on balancing accuracy and computational cost.

Conclusions:

• DDFG offers an effective solution for dynamic coordination in MARL.
• The approach adapts to evolving inter-agent relations, outperforming existing methods.
• DDFG presents a promising direction for tackling complex multi-agent coordination problems.