Optimal Tracking Control of Heterogeneous MASs Using Event-Driven Adaptive Observer and Reinforcement Learning

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Distributed Systems

Background:

• Multiagent systems (MASs) present challenges in coordinating nonidentical agents, especially when information is partially unknown.
• Traditional control methods often require full system knowledge and high communication overhead.
• Reinforcement learning (RL) offers a promising avenue for adaptive control in complex systems.

Purpose of the Study:

• To develop a model-free reinforcement learning (RL) algorithm for output tracking control in nonidentical linear MASs.
• To address scenarios where followers have limited or no prior knowledge of the leader's system information.
• To reduce communication and computational burdens within the MAS.

Main Methods:

• An event-driven adaptive distributed observer is proposed to estimate the leader's system matrix and state.
• An edge-based predictor estimates relative states, and an integral input-based triggering condition manages control input transmission.
• An RL-based state feedback controller is designed, converting the problem into an optimal control problem solved using an off-policy RL algorithm to learn inhomogeneous algebraic Riccati equations (AREs) online.

Main Results:

• The proposed event-driven adaptive observer and RL algorithm enable followers to asymptotically synchronize with the leader's output.
• The off-policy RL algorithm successfully learns the solution to inhomogeneous AREs without system dynamics knowledge.
• Numerical simulations validate the theoretical findings, demonstrating the effectiveness of the proposed approach.

Conclusions:

• The developed model-free RL strategy with an event-driven observer effectively achieves output tracking and synchronization in nonidentical MASs.
• The approach significantly reduces communication and computational load, making it practical for real-world applications.
• This work advances the application of RL in distributed control systems with partial information.