Finite-Horizon Optimal Consensus Control for Unknown Multiagent State-Delay Systems.

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Robotics

Background:

• Optimal consensus control is crucial for coordinating multiagent systems.
• Solving these problems often involves complex, coupled Hamilton-Jacobi-Bellman (HJB) equations.
• Existing methods struggle with unknown system dynamics and state delays.

Purpose of the Study:

• To develop a novel off-policy reinforcement learning (RL) algorithm for finite-horizon optimal consensus control.
• To address challenges posed by unknown dynamics and state delays in multiagent systems.
• To enable learning of optimal control policies using only measurable state data.

Main Methods:

• An off-policy reinforcement learning (RL) algorithm is proposed to learn solutions to time-varying HJB equations.
• Single critic neural networks (NNs) approximate cost functions and derive control policies for each agent.
• Adaptive weight update laws for NNs are derived using the method of weighted residuals.

Main Results:

• The RL algorithm successfully learns two-stage optimal consensus solutions for unknown systems.
• The approach effectively handles state delays without explicit system dynamics knowledge.
• Simulation results demonstrate the efficacy and practical applicability of the proposed method.

Conclusions:

• The developed off-policy RL method provides an effective solution for optimal consensus control in complex multiagent systems.
• This approach offers a data-driven alternative to traditional control methods, especially for systems with unknown dynamics and delays.
• The use of neural networks and adaptive learning laws enhances the robustness and adaptability of the control strategy.