Reinforcement learning-based optimal formation control of underactuated USVs with prescribed performance under...

This paper addresses the optimal formation control problem of underactuated unmanned surface vehicles (USVs) subject to unknown dynamics, communication delays, prescribed performance constraints, environmental disturbances, and input saturation. An integral reinforcement learning-based control scheme is developed. First, the underactuated dynamics are transformed into a standard cascaded integrator form via output redefinition. A prescribed performance control (PPC) scheme with an initial-condition-independent error transformation is then introduced to guarantee full-time formation error constraints. To compensate for communication delays, a data-driven state predictor is designed to estimate neighbors' current states using only delayed information received through the communication network. On this basis, a non-quadratic performance index is constructed, and an optimal control policy is learned online within an integral reinforcement learning (IRL) framework, eliminating the need for an accurate system model. An experience replay (ER) mechanism is incorporated to relax the persistent excitation (PE) requirement and remove the dependence on an initially stabilizing control policy. Lyapunov-based analysis proves that all signals in the closed-loop system are uniformly ultimately bounded (UUB). Finally, simulation results verify the effectiveness and superiority of the proposed control scheme.