Scalable-MADDPG-Based Cooperative Target Invasion for a Multi-USV System.

Area of Science:

• Robotics
• Artificial Intelligence
• Marine Engineering

Background:

• Cooperative target invasion by multiple unmanned surface vehicles (multi-USV) presents scalability challenges.
• Dynamic changes in the number of agents during training can lead to policy instability.

Purpose of the Study:

• To propose a novel scalable deep reinforcement learning (DRL) method for multi-USV cooperative target invasion.
• To address policy oscillation and enhance exploration-exploitation balance in DRL for multi-USV systems.

Main Methods:

• Introduction of Scalable-MADDPG, a reinforcement learning (RL) algorithm allowing real-time system scaling.
• Integration of a bi-directional long-short-term memory (Bi-LSTM) network to stabilize policies.
• Implementation of an improved epsilon-greedy strategy with Ornstein-Uhlenbeck (OU) noise for robust policy optimization.

Main Results:

• The proposed Scalable-MADDPG method demonstrated effective cooperative target invasion for multi-USV systems.
• The Bi-LSTM network successfully mitigated policy oscillations during training.
• The enhanced exploration strategy improved the robustness of the learned policies.

Conclusions:

• Scalable-MADDPG offers a flexible and effective solution for scalable multi-USV cooperative control.
• The integration of Bi-LSTM and an improved exploration strategy enhances DRL performance in complex marine environments.
• The method is validated through experimental results, showing its practical applicability.