Selective entropy-fused proximal policy optimisation with federated reinforcement learning for intelligent multi-UAV...

The rapid evolution of 5G and emerging 6G networks has increased the demand for wireless communication systems that deliver high capacity, low latency, and adaptability. However, conventional terrestrial infrastructure remains costly and inflexible, particularly in dynamic or remote environments. This article develops a new Federated Reinforcement Learning (FRL)-based UAV communication system using Selective Entropy-Fused Proximal Policy Optimization (SEF-PPO) is proposed to enhance the performance of locally-on-policy learning in real-time decision-making environments. In contrast to existing digital twin or offline-trained deep reinforcement learning (DRL) methods, the proposed solution eliminates the need for replay buffers, thereby reducing memory and computational requirements for UAV platforms. UAVs learn collaboratively while preserving data privacy and maintaining robustness to non-IID user distributions through federated aggregation with a High-Altitude Platform (HAP). The framework integrates trajectory planning, user association, energy-efficient resource allocation, and handover management within a unified adaptive architecture. Experimental results demonstrate significant improvements in throughput, fairness, latency, and energy efficiency compared with baseline methods, including DMTD, DRL-EC³, and greedy and random algorithms. Overall, the proposed design enables scalable, energy-aware, and environment-responsive UAV coordination, offering a deployment-ready solution for next-generation wireless networks without requiring simulation-based pretraining.