Path Planning for USVs in Complex Marine Environments Based on an Improved Hybrid TD3 Algorithm

Area of Science:

• Robotics and Autonomous Systems
• Marine Engineering
• Artificial Intelligence

Background:

• Real-time path planning for Unmanned Surface Vehicles (USVs) is complex due to unstructured environments, ocean currents, and dynamic obstacles.
• Existing deep reinforcement learning methods struggle with safety, efficiency, and stability in realistic marine scenarios.

Purpose of the Study:

• To develop an improved deep reinforcement learning algorithm for safe and efficient real-time path planning for USVs.
• To enhance navigation safety, sample efficiency, and policy stability in dynamic marine environments.

Main Methods:

• Formulated path planning as a Markov Decision Process (MDP) with an enhanced state space including perception, currents, and goal information.
• Proposed the Hybrid Safety and Reward-Sensitive Twin Delayed Deep Deterministic Policy Gradient (H_RS_TD3) algorithm.
• Integrated a risk-aware safety architecture, Trajectory Predictor Network (TPN), Curvature-driven Advantage-based Prioritized Experience Replay (CDA-PER), and uncertainty-aware Q-learning.

Main Results:

• The H_RS_TD3 algorithm demonstrated faster convergence and improved policy stability compared to baseline methods.
• Achieved competitive path efficiency with consistent obstacle clearance.
• Maintained millisecond-level inference latency, proving practical feasibility.

Conclusions:

• The proposed H_RS_TD3 algorithm is effective for safe and efficient real-time path planning for USVs in complex, dynamic marine environments.
• The framework enhances navigation safety and computational performance for autonomous marine applications.