DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks

Area of Science:

• Computer Science
• Electrical Engineering
• Oceanography

Background:

• Underwater acoustic sensor networks (UASNs) face significant challenges including environmental dynamics, long propagation delays, and lack of GPS, hindering traditional MAC protocol performance.
• Limited channel utilization and congestion issues impede high data rates in UASNs, necessitating advanced MAC solutions.

Purpose of the Study:

• To develop a novel reinforcement learning (RL) based MAC protocol for UASNs that enhances network throughput and supports asynchronous operation.
• To leverage inherent large propagation delays in underwater environments to improve network performance.

Main Methods:

• A distributed RL-MAC protocol based on framed ALOHA is proposed, enabling nodes to learn optimal transmission strategies (time-slot and transmission-offset selection).
• The protocol learns through environmental interaction, enhancing network resilience and adaptability without requiring external environmental data.
• Performance was evaluated against UW-ALOHA-Q, CS-ALOHA, and DOTS in both static and mobile UASN scenarios.

Main Results:

• The proposed RL-MAC protocol demonstrated significant channel utilization gains: 13%-106% in static and 23%-126% in mobile scenarios.
• Compared to UW-ALOHA-Q, the RL-MAC protocol achieved faster convergence times in larger networks due to a more efficient learning strategy.

Conclusions:

• The RL-MAC protocol effectively addresses UASN challenges, offering substantial improvements in channel utilization and network performance.
• The proposed approach provides a resilient and adaptive solution for medium access control in dynamic underwater acoustic environments.