Reinforcement Learning-Driven Autonomous Path Planning for Unmanned Surface Vehicles: Current Status, Challenges, and...

Area of Science:

• Marine autonomous systems
• Robotics and intelligent control
• Computational intelligence

Background:

• Advancements in marine shipping autonomy necessitate safe navigation for unmanned surface vehicles (USVs) in complex environments.
• Local path planning is crucial for USV autonomous decision-making, requiring real-time collision avoidance and optimization under dynamic and uncertain conditions.
• Reinforcement learning (RL) offers a promising approach for USV path planning due to its adaptability to complex, nonlinear problems and high-dimensional state spaces.

Purpose of the Study:

• To systematically review the literature on reinforcement learning applications for unmanned surface vehicle local path planning.
• To analyze the evolution of RL algorithms and their integration with traditional methods in various water scenarios.
• To identify current research bottlenecks and future development trends in USV path planning.

Main Methods:

• Systematic literature review focusing on RL algorithms like Deep Q-Network (DQN), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), and Twin Delayed Deep Deterministic Policy Gradient (TD3).
• Analysis of hybrid approaches combining RL with traditional path planning algorithms such as A* and Rapidly-exploring Random Tree (RRT).
• Evaluation of method performance, advantages, and limitations in typical marine scenarios.

Main Results:

• RL algorithms demonstrate significant potential for autonomous decision-making in USV local path planning.
• Specific RL algorithms (DQN, PPO, SAC, TD3) and hybrid methods show varying degrees of effectiveness in different scenarios.
• Key challenges identified include reward mechanism design, sample efficiency, sim-to-real transfer, and safety verification.

Conclusions:

• Reinforcement learning is a vital technical route for USV local path planning, but current methods face limitations.
• Future research should focus on enhancing reward mechanisms, improving sample utilization, and ensuring safety and trustworthiness for real-world deployment.
• Data fusion and security enhancement are identified as critical future development directions for robust USV navigation.