Differential Game-Based Deep Reinforcement Learning in Underwater Target Hunting Task.

Area of Science:

• Robotics and Control Systems
• Ocean Engineering
• Game Theory

Background:

• Underwater target hunting is complex due to dynamic ocean environments and adversarial targets.
• Existing methods often neglect environmental factors like currents, winds, and communication delays.
• Coordinated control of multiple unmanned underwater vehicles (UUVs) is crucial for effective hunting missions.

Purpose of the Study:

• To develop an adaptive control scheme for UUVs to ensure consistent target pursuit and prevent escape without collision.
• To leverage differential game theory for analyzing hunter-target interactions.
• To address challenges posed by environmental dynamics and communication constraints in underwater operations.

Main Methods:

• Utilized differential game theory to model adversarial behaviors between UUVs and a high-maneuverability target.
• Conceived a Hamiltonian function with Leibniz's formula to derive feedback control policies.
• Designed a modified multi-agent reinforcement learning (MARL) algorithm incorporating energetic flows and acoustic propagation delays.

Main Results:

• The proposed control policies ensure the target hunting system is asymptotically stable in the mean.
• The system achieves a Nash equilibrium, guaranteeing optimal strategies for all agents.
• The modified MARL scheme demonstrated superior performance compared to typical MARL algorithms in simulations, achieving higher rewards and success rates.

Conclusions:

• The developed control strategy effectively addresses the complexities of underwater target hunting in dynamic and adversarial environments.
• The integration of game theory and adaptive control provides a robust framework for multi-UUV coordination.
• The modified MARL approach offers a promising solution for real-time trajectory scheduling and distributed coordination in challenging underwater scenarios.