Distributed DRL-Based Computation Offloading Scheme for Improving QoE in Edge Computing Environments

Area of Science:

• Computer Science
• Artificial Intelligence
• Distributed Systems

Background:

• Edge computing enhances user experience by processing data closer to the source.
• Reinforcement learning (RL) and deep reinforcement learning (DRL) are used for optimizing edge collaboration.
• Existing DRL schemes lack temporal state consideration and efficient experience utilization in distributed environments.

Purpose of the Study:

• To propose a distributed DRL-based computation offloading scheme for improved Quality of Experience (QoE) in edge computing.
• To address limitations in existing DRL schemes regarding temporal states, experience importance, and data sparsity.

Main Methods:

• Developed a DRL scheme that models task service time and load balance for offloading target selection.
• Incorporated least absolute shrinkage and selection operator (LASSO) regression and an attention layer to account for temporal states.
• Utilized TD error and critic network loss to learn policies based on experience importance.
• Implemented adaptive experience sharing between agents using strategy gradients to mitigate data sparsity.

Main Results:

• The proposed scheme demonstrated lower variation in performance compared to existing methods.
• Achieved higher cumulative rewards, indicating more effective optimization of the offloading policy.
• Successfully addressed temporal state representation and experience utilization challenges in distributed DRL.

Conclusions:

• The novel distributed DRL scheme significantly enhances QoE in edge computing environments.
• The integration of temporal state awareness and experience prioritization leads to superior learning performance.
• Adaptive experience sharing effectively resolves data sparsity issues in distributed DRL for computation offloading.