Deep Reinforcement Learning-Assisted Energy Harvesting Wireless Networks

Area of Science:

• Wireless Communication Networks
• Energy Harvesting Technologies
• Machine Learning Applications

Background:

• Heterogeneous ultra-dense networks (HUDN) face challenges with increasing traffic and power consumption.
• Energy harvesting in HUDN is crucial but hampered by unpredictable harvested energy levels.
• Optimizing energy harvesting and data transmission is vital for network performance.

Purpose of the Study:

• To develop an optimal control strategy for energy harvesting in HUDN.
• To address the challenge of determining when and where to harvest energy from multiple sources.
• To enhance both energy efficiency and data throughput in HUDN.

Main Methods:

• Proposed reinforcement learning methods, specifically Deep Deterministic Policy Gradient (DDPG) and Wolpertinger DDPG (W-DDPG).
• Utilized W-DDPG to manage large and discrete action spaces for controlling base station operations.
• Simulated the proposed algorithms in a HUDN environment with energy harvesting.

Main Results:

• The W-DDPG algorithm demonstrated superior performance compared to the original DDPG and Deep Q-Learning.
• Achieved significant improvements in both energy efficiency and network throughput.
• Validated the effectiveness of W-DDPG in managing random energy harvesting.

Conclusions:

• W-DDPG is an effective approach for optimizing energy harvesting and performance in HUDN.
• Reinforcement learning, particularly W-DDPG, offers a robust solution for dynamic network control.
• The proposed method addresses the unpredictability of harvested energy for improved network operation.