Decentralized queue control with delay shifting in edge-IoT using reinforcement learning.

Area of Science:

• Computer Science
• Electrical Engineering
• Applied Mathematics

Background:

• Edge-IoT systems face challenges with increasing demands for energy efficiency, responsiveness, and self-regulation.
• Unstable traffic conditions in edge networks necessitate adaptive service management strategies.
• Existing models often lack the flexibility to handle dynamic QoS and energy management requirements.

Purpose of the Study:

• To develop an adaptive approach for modeling and managing request service processes at peripheral nodes of edge-IoT systems.
• To enhance energy efficiency, responsiveness, and self-regulation under fluctuating traffic conditions.
• To provide a scalable and traffic-type agnostic solution for dynamic QoS and energy management.

Main Methods:

• A stochastic G/G/1 model with a parameterized time shift was proposed to account for device unavailability.
• Analytical expressions for Quality of Service (QoS) indicators (delay, variability, loss, energy consumption) were derived as functions of the shift parameter.
• A Deep Q-Network (DQN)-based reinforcement learning agent was implemented for decentralized, real-time control of the shift parameter.

Main Results:

• Demonstrated a reduction in average delay by 17-26% compared to state-of-the-art models.
• Achieved decreased fluctuations in service time and improved queue recovery stability after peak loads.
• The proposed solution is traffic-type agnostic and scalable across diverse edge architectures.

Conclusions:

• The adaptive approach effectively models and manages service processes in edge-IoT systems, enhancing QoS and energy efficiency.
• The DQN-based agent provides dynamic, decentralized control, adapting to real-time queue states.
• The findings are applicable to sensor networks, 5G/6G edge scenarios, and systems requiring dynamic QoS and energy management.