Decentralized Consensus Inference-Based Hierarchical Reinforcement Learning for Multiconstrained UAV Pursuit-Evasion...

Area of Science:

• Robotics and Control Systems
• Artificial Intelligence
• Multi-Agent Systems

Background:

• Multi-quadrotor uncrewed aerial vehicle (UAV) systems are crucial for applications like pursuit-evasion games (MC-PEGs).
• Cooperative evasion and formation coverage (CEFC) is a challenging MC-PEG task, especially with limited communication.
• High-dimensional complexities arise from coordinating UAVs amidst obstacles, adversaries, targets, and formation dynamics.

Purpose of the Study:

• To propose a novel two-level framework, consensus inference-based hierarchical reinforcement learning (CI-HRL), for the CEFC task in multi-UAV systems.
• To address challenges in communication-limited environments and high-dimensional problem spaces.
• To enhance the collaborative evasion and task completion capabilities of UAV swarms.

Main Methods:

• Developed a two-level framework: CI-HRL, with a high-level policy for target localization and a low-level policy for navigation and formation.
• Introduced consensus-oriented multiagent communication (ConsMAC) for the high-level policy to enable global perception and consensus from local states.
• Utilized alternative training-based multi-agent deep deterministic policy gradient (AT-M) and policy distillation for the low-level control.

Main Results:

• CI-HRL demonstrated superior performance in enhancing swarm's collaborative evasion and task completion.
• The ConsMAC module effectively enabled agents to aggregate neighbor messages and establish consensus.
• Software-in-the-loop (SITL) simulations validated the framework's effectiveness in complex scenarios.

Conclusions:

• The proposed CI-HRL framework offers a robust solution for challenging multi-UAV cooperative evasion and formation coverage tasks.
• The hierarchical approach with specialized communication and control policies significantly improves swarm performance.
• This research advances the capabilities of multi-agent systems in dynamic and constrained environments.