DT-aided resource allocation via generative adversarial imitation learning in complex cloud-edge-end scenarios

Area of Science:

• Computer Science
• Artificial Intelligence
• Distributed Computing

Background:

• Traditional Deep Reinforcement Learning (DRL) for cloud-edge-end computing resource allocation relies heavily on known parameters and real-time rewards.
• This reliance poses challenges in complex scenarios requiring dynamic decision-making.
• Existing methods struggle with effective service delivery when prior knowledge or feedback is limited.

Purpose of the Study:

• To propose a novel Expert-driven Generative Adversarial Imitation Learning (E-GAIL) model for joint, multi-resource allocation in cloud-edge-end systems.
• To enable efficient resource allocation without requiring prior state knowledge or real-time reward feedback.
• To address the limitations of traditional DRL in complex and dynamic computing environments.

Main Methods:

• Developed a DT-aided E-GAIL model leveraging imitation learning for resource allocation.
• Introduced a single-expert trajectory generation algorithm using Actor-Critic and Noisynet with DT Network historical data.
• Fused multiple single-expert trajectories into a multi-expert trajectory, utilizing Nash equilibrium to resolve conflicts and find optimal solutions.
• Updated E-GAIL generator and discriminator parameters via gradients to match the multi-expert trajectory.

Main Results:

• The E-GAIL Agent rapidly obtains resource allocation policies upon task upload, independent of prior knowledge or real-time rewards.
• Experimental results demonstrate E-GAIL's capability to achieve the best-fit expert trajectory in large-scale, noisy environments.
• The proposed model effectively handles joint allocation of multiple constrained resources.

Conclusions:

• The DT-aided E-GAIL model offers a robust solution for resource allocation in cloud-edge-end computing, especially in complex and data-scarce scenarios.
• E-GAIL overcomes the limitations of traditional DRL by utilizing imitation learning and multi-expert fusion.
• This approach enhances decision-making speed and accuracy in dynamic, large-scale computing environments.