Parametrized Graph Convolutional Multi-Agent Reinforcement Learning with Hybrid Action Spaces in Dynamic Topologies.

Area of Science:

• Artificial Intelligence
• Robotics
• Computational Neuroscience

Background:

• Multi-agent swarm collaboration is vital for dynamic environments.
• Hybrid action spaces in MARL pose challenges due to action coupling, hindering policy stability.
• Current methods struggle to decouple discrete and continuous actions, leading to suboptimal performance.

Purpose of the Study:

• To address the action coupling problem in MARL under dynamic topologies.
• To propose a novel method, Parametrized Graph Convolution Reinforcement Learning (P-DGN), for stable and efficient multi-agent collaboration.
• To investigate a biomimetic observation strategy inspired by starling flocking behavior.

Main Methods:

• Implemented an actor-critic framework with P-DGN, decoupling hybrid action optimization.
• Utilized multi-head attention in the actor network for dynamic relation kernels and Temporal Relation Regularization (TRR).
• Employed a Deep Q-Network (DQN)-based critic for discrete action evaluation and a Gaussian policy for continuous actions.

Main Results:

• P-DGN demonstrated faster convergence and improved training stability compared to P-DQN and DQN baselines.
• Agents trained with P-DGN exhibited emergent cooperative tactics, such as encirclement, under dense rewards.
• The biomimetic observation design, focusing on nearest neighbors, facilitated efficient local interaction and global collaboration.

Conclusions:

• P-DGN offers a robust solution for optimizing hybrid action spaces in MARL within dynamic, open environments.
• The method balances theoretical generality with practical applicability, enhancing swarm collaboration.
• The biomimetic approach provides a biologically plausible framework for advanced multi-agent systems.