Optimal Robust Output Containment of Unknown Heterogeneous Multiagent System Using Off-Policy Reinforcement Learning

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Robotics

Background:

• Cooperative control of multiagent systems (MAS) is crucial for coordinated behavior.
• Achieving robust output containment in MAS with unknown dynamics presents significant challenges.
• Existing model-based approaches often require unavailable nominal system models.

Purpose of the Study:

• To develop a model-free algorithm for the optimal robust output containment problem in general linear heterogeneous MAS.
• To address the limitations of model-based methods that require knowledge of agent dynamics.
• To enable real-time online control without prior system information.

Main Methods:

• Developed an offline policy iteration (PI) algorithm using an internal model principle for known dynamics.
• Introduced a discounted performance function to frame the problem as an output-feedback design with bounded gain.
• Proposed a model-free, off-policy integral reinforcement learning algorithm based on a Bellman equation for online, real-time control.

Main Results:

• The proposed algorithm effectively solves the optimal robust output containment problem for heterogeneous MAS.
• The method operates in real-time without requiring any knowledge of the system dynamics.
• Simulation results demonstrate the successful verification and effectiveness of the proposed approach.

Conclusions:

• The model-free integral reinforcement learning algorithm provides an effective solution for robust output containment in MAS with unknown dynamics.
• This approach overcomes the need for explicit system models, making it applicable to real-world scenarios.
• The findings contribute to advancing autonomous coordination strategies in complex multiagent systems.