VGN: Value Decomposition With Graph Attention Networks for Multiagent Reinforcement Learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Multiagent Systems

Background:

• Cooperative multiagent reinforcement learning problems often involve agents with local observations sharing a joint reward.
• Existing value decomposition methods typically factorize the joint reward but often overlook the graphical relationships between agents.

Purpose of the Study:

• To develop a new value decomposition method that incorporates the dynamical relationships between agents.
• To address the limitations of previous methods by introducing graphical information into the value function decomposition.

Main Methods:

• Introduced a novel value decomposition with graph attention network (VGN) method.
• Designed two graphical neural network-based algorithms, VGN-Linear and VGN-Nonlinear, to solve agent value functions.
• Theoretically proved that the proposed methods satisfy the factorizable condition in centralized training.

Main Results:

• The VGN method allows an agent's decomposition factor to be influenced by related agents' reward signals.
• Evaluated performance on the StarCraft Multiagent Challenge (SMAC) benchmark.
• Demonstrated superior performance compared to state-of-the-art value-based multiagent reinforcement algorithms, particularly on difficult tasks.

Conclusions:

• The proposed VGN method effectively integrates inter-agent graphical information for improved multiagent reinforcement learning.
• VGN offers a significant advancement over existing methods, especially in complex and challenging multiagent scenarios.