Time-Varying HJBE-Based Adaptive Safe Critic Control Design for Stochastic Asymmetric Constrained Multiagent Systems

Area of Science:

• Control Engineering
• Artificial Intelligence
• Systems Science

Background:

• Stochastic multi-agent systems (MASs) present challenges due to asymmetric state and input constraints.
• Designing adaptive and safe control for such systems requires robust methodologies.

Purpose of the Study:

• To develop an adaptive safe critic control design for stochastic MASs with asymmetric constraints.
• To address input limitations and enhance controller robustness against stochastic disturbances.

Main Methods:

• A unified transformation function (UTF) converts constrained problems into unconstrained error systems.
• A nonquadratic cost function handles input limitations.
• A time-varying Hamilton-Jacobi-Bellman equation (HJBE) is formulated using Bellman's principle and Itô's lemma.
• Integral reinforcement learning (IRL) and a time-varying single-critic network approximate HJBE solutions.
• Experience replay (ER) technique enhances learning efficiency and relaxes persistent excitation conditions.

Main Results:

• The proposed method effectively handles asymmetric state and input constraints in stochastic MASs.
• The integral reinforcement learning approach eliminates the need for explicit drift dynamics.
• The single-critic network significantly reduces computational complexity.
• Simulation examples validate the approach's feasibility and effectiveness.

Conclusions:

• The developed adaptive safe critic control framework provides a robust solution for constrained stochastic MASs.
• The integration of IRL and ER offers an efficient and data-driven control strategy.
• The approach demonstrates significant improvements in learning efficiency and computational load.