A Federated Hierarchical DQN-Based Distributed Intelligent Anti-Jamming Method for UAVs

In recent years, with the rapid development of intelligent communication technologies, anti-jamming techniques based on deep learning have been widely adopted in unmanned aerial vehicle (UAV) systems, yielding significant improvements. Most existing studies primarily focus on intelligent anti-jamming decision-making for single UAVs. However, in UAV swarm systems, single-agent decision models often suffer from data isolation and inconsistent frequency usage decisions among nodes within the same task subnet, caused by asynchronous model updates. Although data sharing among UAVs can partially alleviate model update issues, it introduces significant communication overhead and data security challenges. To address these problems, this paper proposes a novel multi-UAV cooperative intelligent anti-jamming decision-making method, termed Federated Learning-Hierarchical Deep Q-Network (FL-HDQN). First, an adaptive model synchronization mechanism is integrated into the federated learning framework. By sharing only local model parameters instead of raw data, UAVs collaboratively train a global model for each task subnet. This approach ensures decision consistency while preserving data privacy and reducing communication costs. Second, to overcome the curse of dimensionality caused by multi-domain interference parameters, a hierarchical deep reinforcement learning model is designed. The model decouples multi-domain optimization into two levels: the first layer performs time-frequency domain decisions, and the second layer conducts power and modulation-coding domain decisions, ensuring both real-time performance and decision effectiveness. Finally, simulation results demonstrate that, compared with state-of-the-art intelligent anti-jamming models, the proposed method achieves 1% higher decision accuracy, validating its superiority and effectiveness.