USV Trajectory Tracking Control Based on Receding Horizon Reinforcement Learning.

Area of Science:

• Robotics
• Control Systems
• Artificial Intelligence

Background:

• Unmanned Surface Vehicles (USVs) require precise trajectory tracking for effective operation.
• Existing control methods like Lyapunov Model Predictive Control (LMPC) and Sliding Mode Control (SMC) have limitations.
• Optimal control in perpetual time domains presents significant challenges.

Purpose of the Study:

• To introduce a novel Receding Horizon Reinforcement Learning (RHRL) approach for high-precision USV trajectory tracking.
• To develop a control architecture combining feedforward and feedback components for enhanced performance.
• To demonstrate the advantages of RHRL over traditional control methods.

Main Methods:

• A composite control architecture with feedforward (path curvature, dynamic model) and feedback (RHRL) components was designed.
• The RHRL algorithm was integrated with a rolling time domain optimization mechanism.
• A time-independent executive-evaluator network was used for value function and control strategy learning within prediction domains.
• Theoretical proofs were provided for RHRL algorithm convergence and closed-loop system stability.

Main Results:

• The RHRL controller provides an explicit state feedback control law, facilitating direct offline and online learning.
• The proposed method effectively converts perpetual optimal control problems into solvable finite time domain problems.
• Simulation tests confirmed the efficacy of the RHRL approach for USV trajectory control.

Conclusions:

• The novel RHRL approach offers a robust and adaptable solution for precise USV trajectory tracking.
• The method's ability for both offline and online learning enhances its practical applicability.
• RHRL demonstrates superior performance and learning capabilities compared to LMPC and SMC.