DQN-empowered energy optimization for wireless powered communication networks

Area of Science:

• Wireless communication networks
• Internet of Things (IoT)
• Sustainable energy systems

Background:

• Wireless Powered Communication Networks (WPCNs) are crucial for powering Internet of Things (IoT) devices.
• Existing energy harvesting models in WPCNs often use linear approximations, leading to inaccuracies due to nonlinear Radio Frequency to Direct Current (RF-DC) conversion saturation.
• Dynamic resource management is essential for optimizing performance and longevity in WPCNs.

Purpose of the Study:

• To propose a Deep Q-Network (DQN)-empowered dynamic resource collaborative management scheme for WPCNs.
• To address the limitations of traditional linear energy harvesting models by incorporating a piecewise nonlinear harvesting model.
• To maximize network utility while balancing energy efficiency and fairness using a Markov Decision Process (MDP) framework.

Main Methods:

• Formulated a multi-objective allocation problem within a Markov Decision Process (MDP) framework.
• Employed a piecewise nonlinear energy harvesting model to accurately capture Radio Frequency to Direct Current (RF-DC) conversion effects.
• Integrated Gaussian Process Regression (GPR) for energy harvest prediction within a closed-loop optimization system.
• Provided theoretical convergence proofs for Q-learning and Lyapunov stability analysis for energy queue errors.

Main Results:

• Extended network lifetime by 56.4% (117 to 183 rounds).
• Reduced energy allocation standard deviation by 56.8% (23.7 mJ to 12.3 mJ).
• Improved convergence speed by 53.1% (150 vs. 320 episodes) and dynamic adaptability by 66.7% (5 vs. 15 rounds).
• Increased network throughput by 33.33% (80 vs. 60 Mbps).

Conclusions:

• The proposed DQN-empowered scheme significantly enhances WPCN performance and sustainability.
• The nonlinear harvesting model and MDP framework provide a more accurate and efficient approach to resource management.
• The findings support the large-scale deployment of WPCNs for future IoT applications.