Reinforcement-Learning-Based Fuzzy Bipartite Consensus for Multiagent Systems: A Novel Scaling Off-Policy Learning...

Area of Science:

• Control Theory
• Artificial Intelligence
• Systems Engineering

Background:

• Nonlinear multiagent systems (NMASs) present challenges in achieving coordinated behavior.
• Distributed control for NMASs often requires complete knowledge of system dynamics, which is frequently unavailable.
• Bipartite consensus (BC) is a critical objective for specific network structures in NMASs.

Purpose of the Study:

• To investigate the bipartite consensus (BC) problem for nonlinear multiagent systems (NMASs) with unknown system dynamics.
• To develop a distributed control strategy that does not rely on explicit knowledge of system dynamics.
• To reformulate the BC problem as a solvable zero-sum game.

Main Methods:

• Representing NMAS dynamics using the Takagi-Sugeno (T-S) fuzzy model.
• Introducing a minmax game policy to achieve distributed control.
• Reformulating the BC problem as a zero-sum game solvable via game algebraic Riccati equations (GAREs).
• Proposing a novel scaling off-policy iteration (PI) algorithm to solve GAREs without system dynamics or initial policies.

Main Results:

• The proposed scaling PI algorithm relaxes the reliance on system dynamics during learning.
• The algorithm eliminates the need for initial admissible control policies, unlike traditional PI methods.
• Faster convergence speeds are achieved compared to standard value iteration techniques.
• The method's effectiveness is demonstrated through simulations and comparative experiments.

Conclusions:

• The developed approach effectively solves the bipartite consensus problem for NMASs with unknown dynamics.
• The novel scaling PI algorithm offers advantages in terms of data requirements and convergence speed.
• This work provides a robust framework for distributed control design in complex multiagent systems.