Dynamic threshold-enhanced diffusion PPO for multi-UAV collaborative optimization in wireless rechargeable sensor...

Area of Science:

• Computer Science
• Electrical Engineering
• Network Optimization

Background:

• Multi-Unmanned Aerial Vehicle (UAV) systems face challenges in optimizing communication delay and load balancing.
• Wireless Rechargeable Sensor Networks (WRSNs) require efficient management for sustained operation.
• Existing algorithms struggle with the dynamic and complex nature of these networks.

Purpose of the Study:

• To develop a robust algorithm for collaborative optimization of communication delay and UAV load balancing in multi-UAV-assisted WRSNs.
• To enhance the feasibility and practicality of optimization solutions by incorporating multi-dimensional constraints.
• To improve the exploration ability and training stability of reinforcement learning algorithms in this domain.

Main Methods:

• Constructed a multi-objective optimization model for multi-UAV-assisted WRSNs.
• Designed a Markov Decision Process (MDP) framework with dynamic weighting for objective balancing.
• Integrated a diffusion model into the Proximal Policy Optimization (PPO) policy network for action diversification.
• Implemented a dynamic threshold strategy based on normalized reward change rate for real-time policy updates.

Main Results:

• The proposed Dynamic Threshold-enhanced Diffusion Proximal Policy Optimization (DTD-PPO) algorithm effectively balances communication delay and UAV load.
• The diffusion model integration enhanced exploration and training stability.
• The dynamic threshold strategy allowed for adaptive policy updates.
• Validated effectiveness using metrics including data collection delay, UAV flight distance deviation, and energy efficiency.

Conclusions:

• The DTD-PPO algorithm demonstrates superior performance and robustness compared to benchmark methods.
• The proposed approach offers a significant advancement in optimizing complex multi-UAV-assisted WRSNs.
• This work provides a practical and effective solution for enhancing the performance of UAV-assisted networks.