Leader-follower UAVs formation control based on a deep Q-network collaborative framework.

Area of Science:

• Robotics
• Artificial Intelligence
• Aerospace Engineering

Background:

• Unmanned Aerial Vehicles (UAVs) face significant challenges in dynamic and uncertain flight environments.
• Coordinated control of multi-UAV systems is crucial for complex missions.
• Existing control methods struggle with the high dynamism and uncertainty inherent in UAV operations.

Purpose of the Study:

• To develop and validate a collaborative framework for regulating leader-follower UAV formations.
• To address the control challenges posed by dynamic and uncertain UAV flight environments.
• To enhance the collective state regulation of UAV swarms using intelligent control.

Main Methods:

• Development of a dynamic model capturing the collective state of UAV formations (relative positions, angles, velocities).
• Application of the Markov Decision Process (MDP) framework to model UAV collaborative operations.
• Utilization of Reinforcement Learning (RL), specifically the Deep Q-Network (DQN) algorithm, for control.
• Implementation of a [Formula: see text]-imitation technique for action selection within the DQN framework.

Main Results:

• Numerical simulations validated the efficacy and portability of the DQN-based control approach.
• The average reward curve demonstrated satisfactory convergence, indicating successful learning.
• The kinematic links between formation nodes met essential controller requirements, ensuring formation integrity.

Conclusions:

• The proposed DQN-based collaborative framework effectively regulates leader-follower UAV formations in challenging environments.
• The intelligent control strategy ensures stable kinematic relationships within the UAV swarm.
• The study substantiates the robustness and applicability of deep reinforcement learning for advanced UAV coordination.