Selective entropy-fused proximal policy optimisation with federated reinforcement learning for intelligent multi-UAV...

Area of Science:

• Wireless Communication Systems
• Artificial Intelligence
• Network Engineering

Background:

• Terrestrial infrastructure for 5G/6G wireless networks is costly and inflexible.
• Unmanned Aerial Vehicle (UAV) communication systems require high capacity, low latency, and adaptability.
• Existing digital twin or offline-trained deep reinforcement learning (DRL) methods have limitations in memory and computation for UAVs.

Purpose of the Study:

• To develop a Federated Reinforcement Learning (FRL)-based UAV communication system.
• To enhance real-time decision-making performance using Selective Entropy-Fused Proximal Policy Optimization (SEF-PPO).
• To enable collaborative UAV learning while preserving data privacy and ensuring robustness to non-IID user distributions.

Main Methods:

• Implementation of a novel FRL framework integrating trajectory planning, user association, resource allocation, and handover management.
• Utilizing SEF-PPO for locally-on-policy learning, eliminating the need for replay buffers.
• Federated aggregation with a High-Altitude Platform (HAP) for collaborative learning and privacy preservation.

Main Results:

• Significant improvements in throughput, fairness, latency, and energy efficiency compared to baseline methods (DMTD, DRL-EC³, greedy, random).
• Reduced memory and computational requirements for UAV platforms.
• Demonstrated scalability, energy awareness, and environment responsiveness in UAV coordination.

Conclusions:

• The proposed FRL-based UAV communication system offers a deployment-ready solution for next-generation wireless networks.
• The system provides an adaptive architecture for dynamic and remote environments.
• It enables efficient and robust UAV coordination without requiring simulation-based pretraining.