A Novel Medium Access Policy Based on Reinforcement Learning in Energy-Harvesting Underwater Sensor Networks

Area of Science:

• Marine Technology
• Sensor Networks
• Energy Harvesting

Background:

• Underwater acoustic sensor networks (UASNs) are crucial for sub-sea exploration and long-term missions.
• Battery dependency limits the operational lifetime of underwater wireless sensor networks.
• Efficient energy management is vital for sustained underwater operations.

Purpose of the Study:

• To maximize harvested energy in UASNs by optimizing TDMA time slot scheduling.
• To prolong the operational lifetime of underwater sensors through intelligent energy management.
• To investigate the impact of piezoelectric energy harvesting and spatial resource uncertainty on network performance.

Main Methods:

• A stochastic model for piezoelectric energy harvesting was examined, considering spatial uncertainty of underwater resources.
• A novel multi-agent reinforcement learning (MARL) algorithm was proposed for autonomous TDMA time slot scheduling.
• The MARL algorithm enables sensor nodes to adapt communication slots based on real-time energy harvesting conditions.

Main Results:

• Piezoelectric energy harvesting improved network lifetime metrics: 4% for First Node Dead (FND), 14% for Half Node Dead (HND), and 22% for Last Node Dead (LND).
• The harvesting-aware TDMA-RL method further boosted HND by 17% and LND by 38%.
• The proposed method enhances in-cluster communication time interval utilization, outperforming traditional methods in throughput and energy harvesting efficiency.

Conclusions:

• The developed harvesting-aware TDMA-RL approach significantly extends the operational lifetime of underwater acoustic sensor networks.
• Autonomous adaptation of communication slots based on energy harvesting conditions is key to improving network efficiency.
• This research provides a robust solution for energy management in battery-dependent underwater wireless sensor networks.