Graph Soft Actor-Critic Reinforcement Learning for Large-Scale Distributed Multirobot Coordination.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Learning distributed cooperative policies for large-scale multirobot systems is a significant challenge in multi-agent reinforcement learning (MARL).
• Existing methods often struggle with scalability and efficient coordination in complex, dynamic environments.

Purpose of the Study:

• To propose a novel Graph Neural Network (GNN)-based algorithm for training distributed cooperative policies in multirobot systems.
• To address the challenges of scalability and sample efficiency in MARL for coordination tasks.

Main Methods:

• Developed a novel off-policy actor-critic MARL algorithm, Graph Soft Actor-Critic (G-SAC), leveraging GNNs to model robot interactions as a graph.
• Designed a GNN-parameterized Gaussian policy for distributed decision-making in continuous action spaces.
• Introduced a scalable, GNN-based value function decomposition technique for a centralized critic network.

Main Results:

• G-SAC demonstrated strong sample efficiency and scalability in custom multirobot coordination environments.
• The trained policies exhibited robust zero-shot generalization capabilities in large-scale coordination problems.
• Empirical results validated the effectiveness of the GNN-based approach for distributed MARL.

Conclusions:

• The proposed G-SAC algorithm effectively trains distributed cooperative policies for large-scale multirobot systems.
• GNNs provide a powerful mechanism for information extraction and coordination in MARL.
• The approach offers a promising direction for advancing multirobot coordination and learning.