Federated deep reinforcement learning based trajectory design for UAV-assisted networks with mobile ground devices

Area of Science:

• Wireless Communication
• Robotics
• Artificial Intelligence

Background:

• Unmanned aerial vehicles (UAVs) offer efficient wireless data collection but face challenges like trajectory planning, privacy, and complex environments with mobile ground devices (GDs).
• Existing methods struggle with dynamic environments and nonconvex optimization problems arising from no-fly zones and collision avoidance.

Purpose of the Study:

• To minimize the overall operation time cost in a multi-UAV assisted data collection system.
• To jointly optimize the 3D trajectories of UAVs and the communication scheduling of GDs.
• To address challenges posed by no-fly zones, collision avoidance, and the dynamic movement of GDs.

Main Methods:

• The problem was transformed into a Markov decision process.
• A multi-agent federated reinforcement learning (MAFRL) approach was proposed for dynamic optimization.
• A multi-step propagation technique and dueling network architecture were used to enhance agent training and stability.

Main Results:

• The MAFRL-based approach effectively optimizes UAV trajectories and GD communication scheduling.
• The proposed method demonstrates improved convergence rate and overall stability in dynamic scenarios.
• Experimental results validate the effectiveness of the approach in practical data collection scenarios.

Conclusions:

• The MAFRL approach provides an effective solution for dynamic UAV-assisted data collection.
• Joint optimization of UAV trajectories and GD scheduling is crucial for efficient operation.
• The study highlights the potential of advanced reinforcement learning techniques in complex wireless systems.