Adaptive Optimal Tracking Control of an Underactuated Surface Vessel Using Actor-Critic Reinforcement Learning

Area of Science:

• Marine Engineering
• Control Systems
• Robotics

Background:

• Underactuated surface vessels face challenges from modeling uncertainties and external disturbances.
• Existing control methods may not adequately address these complex dynamics for optimal tracking.

Purpose of the Study:

• To develop an adaptive reinforcement learning optimal tracking control (RLOTC) algorithm for underactuated surface vessels.
• To enhance control robustness and achieve optimal tracking performance despite uncertainties and disturbances.

Main Methods:

• Integration of backstepping technique with optimized control design.
• Utilization of neural network (NN) approximators for uncertain dynamics.
• Application of adaptive control for disturbance estimation.
• Employing actor-critic networks within a reinforcement learning framework to solve Hamilton-Jacobi-Bellman equations.

Main Results:

• The RLOTC algorithm ensures semiglobal uniform ultimate boundedness of closed-loop systems via Lyapunov stability theorem.
• Synchronous training of adaptive parameters for actor-critic networks, NN approximators, and adaptive control.
• Demonstrated compensation for uncertain vessel dynamics and unknown disturbances.

Conclusions:

• The proposed RLOTC algorithm offers superior, optimized control performance compared to existing methods.
• The approach effectively handles uncertainties and disturbances in underactuated surface vessel control.
• Simulation studies validate the algorithm's effectiveness for underactuated surface vessels.