Autonomous Path Planning for USV Swarm Based on Dual-Module Learning MATD3

The complex and dynamic maritime environment brings significant uncertainties to autonomous path planning tasks, posing substantial challenges for multiagent reinforcement learning (MARL). To address these challenges, this article analyzes the kinematic and dynamic models of unmanned surface vessels (USVs) as well as the environmental disturbances (e.g., wind, waves, and currents), and then proposes a dual-module learning multiagent twin delayed deep deterministic (DML-MATD3) policy gradient framework for USV swarm path planning based on the realistic physical conditions. The framework establishes a motion-generation module and a collision-avoidance module to enable simpler yet more effective reward designs for learning. Specifically, tailored reward functions are independently designed for each module. A potential field method (PFM) is introduced to provide dense and informative guidance for both motion-generation and collision-avoidance modules. Moreover, an adaptive energy consumption reward (AECR) is integrated into the motion-generation module to improve energy-efficient navigation under environmental disturbances. To further enhance exploration efficiency and reward responsiveness during early training, an Ornstein-Uhlenbeck noise-based action enhancement strategy (OU-AES) is employed. Extensive experiments against seven baseline algorithms demonstrate that the proposed DML-MATD3 consistently achieves faster convergence, improved training stability, shorter path lengths, reduced task execution times, and superior overall performance in complex maritime environments.