Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning

Area of Science:

• Computer Science
• Artificial Intelligence
• Network Engineering

Background:

• The proliferation of Internet of Things (IoT) devices strains network bandwidth, necessitating efficient management strategies.
• Smart environments present complex and uncertain conditions for emergency bandwidth distribution, making predefined rules inadequate.
• Traditional reinforcement learning (RL) offers a solution for adaptive control in such environments without requiring a supervisor.

Purpose of the Study:

• To enhance bandwidth management in smart environments with limited shared bandwidth for IoT devices.
• To overcome the limitations of fixed threshold constraints in fog fragment cooperation for emergency bandwidth allocation.
• To introduce and evaluate a hierarchical reinforcement learning (HRL) approach for adaptive and efficient bandwidth management.

Main Methods:

• Implementation of a two-level hierarchical reinforcement learning (HRL) framework.
• The first level learns the optimal adaptive threshold for bandwidth allocation over time.
• The second level enables fog nodes to select the most suitable device for temporary bandwidth lending during emergencies.

Main Results:

• The HRL approach successfully removes the restriction of a fixed threshold, allowing for dynamic adaptation.
• Fog nodes effectively learn to cooperate and lend bandwidth to devices in need.
• Despite increased learning complexity, the HRL method demonstrates improved efficiency in terms of time and overall performance.

Conclusions:

• Hierarchical reinforcement learning (HRL) provides a more sophisticated and effective solution for bandwidth management in dynamic IoT environments.
• The adaptive threshold and intelligent fog node cooperation significantly enhance network performance during emergencies.
• This approach offers a robust framework for addressing bandwidth constraints in the evolving landscape of smart environments.