Adaptive optimal trajectory tracking control of AUVs based on reinforcement learning

Area of Science:

• Robotics and Control Systems
• Artificial Intelligence
• Ocean Engineering

Background:

• Optimal control for Autonomous Underwater Vehicles (AUVs) typically involves solving the complex Hamilton-Jacobi-Bellman (HJB) equation.
• AUV dynamics present nonlinearities and complexities that make direct HJB equation solutions challenging.
• Input saturation is a common constraint in AUV control systems, requiring specific handling.

Purpose of the Study:

• To propose an adaptive, model-free optimal reinforcement learning (RL) neural network (NN) control scheme for AUV trajectory tracking.
• To address the challenges of solving the HJB equation for AUVs by employing an actor-critic RL framework.
• To develop a novel controller design based on filtering errors for simplified control and faster system response in AUVs with second-order strict-feedback dynamics.

Main Methods:

• An actor-critic reinforcement learning (RL) framework with neural networks (NNs) is used to approximate the HJB equation solution.
• A filtering error-based optimal controller design is introduced for AUVs with second-order strict-feedback dynamics.
• An extended state observer (ESO) estimates unknown nonlinear dynamics, and adaptive laws estimate unknown parameters, while an auxiliary variable system handles input saturation.

Main Results:

• The proposed method successfully approximates the optimal control solution without requiring an explicit AUV model.
• The filtering error-based approach simplifies controller design and improves system response speed.
• Strict Lyapunov analysis confirms that all system signals are semi-globally uniformly ultimately bounded (SGUUB).

Conclusions:

• The developed adaptive model-free optimal RL control scheme effectively addresses trajectory tracking for AUVs with input saturation.
• The novel filtering error-based controller design offers advantages in simplicity and performance.
• Comparative experiments validate the superiority of the proposed method over existing approaches.