Autonomous Path Planning for USV Swarm Based on Dual-Module Learning MATD3.

Area of Science:

• Robotics
• Artificial Intelligence
• Marine Engineering

Background:

• Autonomous navigation in complex maritime environments presents significant challenges for multiagent reinforcement learning (MARL).
• Unmanned Surface Vessels (USVs) require robust path planning to handle uncertainties like wind, waves, and currents.
• Existing MARL approaches struggle with the dynamic and unpredictable nature of the marine setting.

Purpose of the Study:

• To propose a novel Dual-Module Learning Multiagent Twin Delayed Deep Deterministic (DML-MATD3) policy gradient framework for USV swarm path planning.
• To enhance path planning by considering USV kinematic/dynamic models and environmental disturbances.
• To improve learning efficiency and navigation performance in realistic maritime conditions.

Main Methods:

• Developed a DML-MATD3 framework with distinct motion-generation and collision-avoidance modules.
• Integrated a Potential Field Method (PFM) for guidance and an Adaptive Energy Consumption Reward (AECR) for efficiency.
• Employed an Ornstein-Uhlenbeck noise-based Action Enhancement Strategy (OU-AES) to boost early training exploration.
• Designed tailored reward functions for each module to simplify learning.

Main Results:

• The DML-MATD3 framework demonstrated faster convergence and improved training stability compared to seven baseline algorithms.
• Achieved shorter path lengths and reduced task execution times in complex maritime environments.
• Showcased superior overall performance in autonomous path planning for USV swarms.

Conclusions:

• The proposed DML-MATD3 framework effectively addresses the challenges of autonomous path planning for USV swarms in dynamic maritime settings.
• The dual-module design and integrated methods enhance navigation efficiency, stability, and energy awareness.
• This approach offers a promising solution for robust multiagent reinforcement learning in real-world marine applications.