A Comparative Study of Control Approaches in Hybrid Reinforcement Learning-Based Drone Swarms

Area of Science:

• Artificial Intelligence
• Robotics
• Control Systems

Background:

• Reinforcement learning (RL) is crucial for autonomous coordination in multi-unmanned aerial vehicle (UAV) systems.
• The impact of control-level action implementation on RL policy effectiveness in multi-UAV coordination is understudied.
• Cooperative surveillance missions require robust and efficient UAV coordination strategies.

Purpose of the Study:

• To comparatively analyze the influence of different control methods on RL-based multi-UAV system performance.
• To evaluate how action abstractions (heading-based, waypoint-based, deterministic) affect learning efficiency, robustness, and operational outcomes in cooperative surveillance.
• To provide practical guidelines for selecting control strategies in aerial swarm applications.

Main Methods:

• A unified hybrid-AI architecture was employed, keeping the RL policy structure consistent across configurations.
• Three control methods—heading-based, waypoint-based, and deterministic—were isolated as the sole variable.
• Statistically rigorous Monte Carlo simulations and non-parametric hypothesis testing were used for evaluation.

Main Results:

• Heading-based control demonstrated superior performance in revisit period, target acquisition time, and tracking continuity.
• Performance gains were attributed to enhanced reactivity and constraint handling, not policy learning differences.
• The study confirmed the critical role of control-level design in RL-based multi-agent systems.

Conclusions:

• Heading-based control is a highly effective action abstraction for RL-based multi-UAV cooperative surveillance.
• Optimizing control-level design is essential for maximizing the performance of autonomous coordination systems.
• The findings offer practical insights for deploying UAV swarms in complex operational environments.