Constrained Soft Actor-Critic for Joint Computation Offloading and Resource Allocation in UAV-Assisted Edge...

Area of Science:

• Computer Science
• Artificial Intelligence
• Wireless Communications

Background:

• Unmanned Aerial Vehicle (UAV)-assisted edge computing is crucial for latency-sensitive applications.
• Optimal resource allocation under strict latency and stochastic channel conditions is a significant challenge.

Purpose of the Study:

• To address the joint computation partitioning and power allocation problem in UAV-assisted edge computing.
• To develop a robust algorithm that explicitly models and satisfies latency constraints.

Main Methods:

• Formulated the problem as a Constrained Markov Decision Process (CMDP).
• Proposed Constrained Soft Actor-Critic (C-SAC), a deep reinforcement learning algorithm combining maximum-entropy policy optimization with Lagrangian dual methods.
• Utilized a dedicated constraint critic network and an adaptive Lagrange multiplier.

Main Results:

• C-SAC achieved an 18.9% constraint violation rate, a substantial improvement over existing methods.
• Demonstrated strong channel-adaptive policies with a -0.894 correlation between local computation ratio and channel quality.
• Showcased robust performance with minimal violation rate increase even with tripled channel variability.

Conclusions:

• Constrained reinforcement learning is a viable approach for reliable UAV edge computing.
• C-SAC effectively balances energy efficiency and latency satisfaction without manual tuning.
• The algorithm ensures stringent quality-of-service requirements are met in dynamic environments.