Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS With Unidentified Exosystem...

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Robotics

Background:

• Heterogeneous multiagent systems (HMASs) present control challenges due to diverse agent dynamics.
• Existing methods often require knowledge of the exosystem's dynamics, limiting applicability.
• Distributed control strategies are crucial for scalability and robustness in HMASs.

Purpose of the Study:

• To develop a data-efficient, off-policy reinforcement learning (RL) approach for distributed output tracking control in HMASs.
• To address scenarios where the exosystem dynamics are completely unknown to all agents.
• To enable adaptive control and state estimation in a fully distributed manner.

Main Methods:

• An identifiable algorithm with experience replay is used to identify system matrices of a novel reference model.
• An output-based, distributed adaptive output observer estimates leader states with low data transmission.
• A data-efficient RL algorithm designs optimal controllers offline using system trajectories.
• Approximate dynamic programming (ADP) iteratively solves game algebraic Riccati equations (GAREs) using online data.

Main Results:

• The proposed method effectively achieves distributed output tracking control for HMASs with unknown exosystem dynamics.
• The adaptive output observer provides accurate leader state estimations with reduced communication overhead.
• The RL-based controller design is data-efficient, avoiding the need for explicit output regulator equations.
• The ADP approach relaxes the requirement for prior knowledge of agent system matrices.

Conclusions:

• The developed data-efficient RL and ADP framework offers a robust solution for distributed control of HMASs.
• The approach enhances control performance and adaptability in systems with unknown dynamics.
• The proposed observer and controller design are suitable for practical implementation in complex multiagent systems.