Multi-UAV Escape Target Search: A Multi-Agent Reinforcement Learning Method.

Area of Science:

• Robotics and Autonomous Systems
• Artificial Intelligence
• Decision Making

Background:

• Multi-agent reinforcement learning (MARL) is key for autonomous Unmanned Aerial Vehicle (UAV) decision-making in complex environments.
• Existing target probability maps are less effective when targets can escape, reducing search efficiency.
• Dynamic escape behaviors of targets challenge traditional multi-UAV search strategies.

Purpose of the Study:

• To address the challenge of multi-UAV target search with static obstacles and dynamic, escaping targets.
• To develop an effective algorithm for decentralized, partially observable environments.
• To enhance the efficiency and success rate of multi-UAV search operations.

Main Methods:

• Modeling the problem as a decentralized partially observable Markov decision process.
• Proposing a spatio-temporal efficient exploration network.
• Introducing a global convolutional local ascent mechanism.
• Developing the multi-UAV Escape Target Search algorithm based on MAPPO (ETS-MAPPO).

Main Results:

• The ETS-MAPPO algorithm demonstrated superior performance compared to five classic MARL algorithms.
• Significant improvements were observed in the number of successful target searches.
• Enhanced area coverage rates were achieved by the proposed ETS-MAPPO algorithm.

Conclusions:

• The ETS-MAPPO algorithm effectively tackles the difficulty of searching for escaping targets in dynamic environments.
• The proposed methods offer a more efficient and robust solution for multi-UAV search missions.
• This research advances autonomous decision-making capabilities for UAV swarms in challenging scenarios.