Research on reinforcement learning-based safe decision-making methodology for multiple unmanned aerial vehicles

Area of Science:

• Robotics and Artificial Intelligence
• Control Systems Engineering
• Autonomous Systems

Background:

• Multi-unmanned aerial vehicle (multi-UAV) systems offer significant advantages for complex tasks.
• Deep reinforcement learning (DRL) shows promise for multi-UAV decision-making but faces challenges in safety and training efficiency.
• Existing DRL methods require improvement for practical application in safety-critical multi-UAV operations.

Purpose of the Study:

• To develop a novel decision-making framework for multi-UAV systems that enhances both safety and training efficiency.
• To address the limitations of current DRL approaches in real-world multi-UAV deployments.
• To enable cooperative and adaptive control strategies for multi-UAVs in dynamic environments.

Main Methods:

• A transfer-safe soft actor-critic (TSSAC) algorithm is proposed for multi-UAV decision-making.
• Each UAV's decision-making is modeled as a constrained Markov decision process (CMDP) to incorporate safety constraints.
• The soft actor-critic-Lagrangian (SAC-Lagrangian) algorithm is integrated with a modified Lagrangian multiplier within the CMDP framework.
• Parameter-based transfer learning is employed to facilitate cooperative and efficient multi-UAV task training.

Main Results:

• Simulation experiments demonstrate that the TSSAC method significantly improves safety during multi-UAV operations.
• The proposed approach enhances the efficiency of training multi-UAV systems.
• The developed method enables UAVs to effectively adapt to dynamic and changing operational scenarios.
• The integration of CMDP and SAC-Lagrangian ensures that safety constraints are met while maximizing task performance.

Conclusions:

• The transfer-safe soft actor-critic (TSSAC) algorithm provides a viable solution for safe and efficient decision-making in multi-UAV systems.
• The proposed method effectively balances task performance with safety constraints through a modified CMDP formulation.
• Parameter-based transfer learning accelerates cooperative training and improves the adaptability of multi-UAVs.
• This research contributes to the practical deployment of DRL in complex, safety-critical autonomous systems.