Data-Based Optimal Consensus Control for Multiagent Systems With Policy Gradient Reinforcement Learning.

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Distributed Computing

Background:

• Multiagent systems often face challenges with unknown dynamics and differing computational capabilities among agents.
• Achieving consensus control in these complex systems requires robust and adaptive learning algorithms.
• Existing methods may struggle with real-time performance and asynchronous learning scenarios.

Purpose of the Study:

• To develop a data-based distributed control algorithm for discrete-time multiagent systems with unknown dynamics.
• To address challenges posed by computational ability differences and ensure asynchronous learning.
• To guarantee the convergence, stability, and optimality of the proposed control strategies.

Main Methods:

• A data-based distributed control algorithm using offline system interaction data sets.
• Distributed policy gradient reinforcement learning (RL) for policy improvement.
• Functional analysis and Lyapunov method for convergence and stability guarantees.
• An asynchronous extension to handle varying computational abilities.
• An actor-critic neural network structure with the method of weighted residuals.

Main Results:

• The proposed algorithm ensures real-time performance and improves system performance using interactive data.
• Convergence and stability are mathematically guaranteed for the distributed control system.
• The asynchronous version effectively handles differing agent computational speeds.
• The actor-critic networks demonstrate convergence and optimality, with approximation errors tending to zero.
• Simulations validate the effectiveness of the developed algorithm.

Conclusions:

• The data-based distributed reinforcement learning approach provides an effective solution for consensus control in complex multiagent systems.
• The algorithm is robust to unknown system dynamics and computational heterogeneity.
• The work advances the field of distributed control by enabling stable and optimal performance in asynchronous learning environments.