Optimized distributed formation control using identifier-critic-actor reinforcement learning for a class of...

Area of Science:

• Robotics
• Control Theory
• Artificial Intelligence

Background:

• Multi-agent systems (MAS) often face challenges with unknown dynamics and stochastic environments.
• Traditional reinforcement learning (RL) for optimal control can be complex and difficult to apply to stochastic systems due to reliance on the Hamilton-Jacobi-Bellman (HJB) equation.

Purpose of the Study:

• To propose an adaptive reinforcement learning (RL)-based optimized distributed formation control strategy.
• To address the challenge of unknown dynamics in stochastic nonlinear single-integrator multi-agent systems (MAS).

Main Methods:

• An adaptive identifier neural network (NN) is employed to estimate the unknown dynamics of the stochastic MAS.
• A reinforcement learning (RL) approach, utilizing actor and critic neural networks, is implemented for optimized formation control.
• The adaptive RL laws are derived from a positive function, simplifying the algorithm compared to methods based on the HJB equation's negative gradient.

Main Results:

• The proposed method successfully identifies unknown system dynamics under expectation.
• The adaptive RL framework enables optimized distributed formation control for the multi-agent system.
• Theorem proofs and computer simulations validate the effectiveness of the optimized control scheme.

Conclusions:

• The developed adaptive RL-based formation control is effective for unknown stochastic nonlinear multi-agent systems.
• The simplified RL algorithm allows for smoother implementation in stochastic dynamical systems.
• The method achieves the desired control objectives for distributed formation control.