Deep Reinforcement Learning-Based Path-Following Control for Underactuated Autonomous Underwater Vehicles

Area of Science:

• Robotics
• Artificial Intelligence
• Ocean Engineering

Background:

• Autonomous Underwater Vehicles (AUVs) require robust path-following control.
• Environmental disturbances and model uncertainties pose significant challenges.
• Existing control methods often struggle with dynamic ocean conditions.

Purpose of the Study:

• To develop a model-free deep reinforcement learning framework for AUV path-following.
• To enhance control accuracy and robustness against ocean currents and model uncertainties.
• To validate the proposed framework through simulations and physical experiments.

Main Methods:

• Integration of an integral line-of-sight (LOS) guidance law with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm.
• Utilizing an LSTM network for temporal dependencies and treating LOS look-ahead distance as a learnable variable.
• Employing a progressive unfreezing transfer learning strategy with attention-based feature-current fusion for domain adaptation.

Main Results:

• ILLT reduced average cross-track error by 48.5% compared to ILT and 66.4% compared to PID control.
• Achieved significantly faster convergence in target domains compared to baseline methods.
• Physical experiments confirmed feasibility and robustness, with tracking errors close to simulation results.

Conclusions:

• The proposed ILLT framework effectively addresses challenges in AUV path-following control.
• The model-free approach demonstrates superior performance and robustness in dynamic environments.
• ILLT offers a promising solution for underactuated AUV navigation tasks.