Deep Reinforcement Learning for Edge Service Placement in Softwarized Industrial Cyber-Physical System

Area of Science:

• Computer Science
• Electrical Engineering
• Industrial Systems

Background:

• Industrial cyber-physical systems (CPS) require efficient processing of delay-sensitive services at the network edge.
• Limited edge resources necessitate optimized service placement and resource allocation strategies.
• Existing solutions inadequately address joint optimization of service placement, workload scheduling, and resource allocation under uncertain demands.

Purpose of the Study:

• To address the limitations in current industrial CPS service placement strategies.
• To minimize service response delay by jointly optimizing service placement, workload scheduling, and resource allocation.
• To develop a robust algorithm capable of handling uncertain service demands.

Main Methods:

• Formulation of a joint optimization problem to minimize service response delay.
• Development of an improved deep Q-network (DQN)-based algorithm for service placement.
• Integration of convex optimization for optimal resource allocation, guided by DQN for placement and scheduling decisions.

Main Results:

• The proposed DQN-based algorithm effectively optimizes service placement, workload scheduling, and resource allocation.
• Experimental results demonstrate a significant reduction in average service response time compared to existing algorithms.
• The algorithm achieves an 8-10% decrease in average service response time.

Conclusions:

• The developed DQN-based approach offers a superior solution for service placement in industrial CPS.
• This method effectively minimizes service response delay by optimizing resource allocation and workload scheduling.
• The findings provide a valuable framework for enhancing the performance of edge computing in industrial environments.