RL-PMO: A Reinforcement Learning-Based Optimization Algorithm for Parallel SFC Migration.

Area of Science:

• Computer Science
• Network Engineering
• Artificial Intelligence

Background:

• Edge networks face disruptions from hardware failures and resource constraints.
• Service Function Chains (SFCs) are vulnerable to node failures, necessitating efficient migration strategies.
• Migrating multiple Virtual Network Functions (VNFs) simultaneously under limited resources is a significant challenge.

Purpose of the Study:

• To propose an efficient and reliable algorithm for parallel migration of multiple VNFs in edge networks.
• To address the challenges of resource limitations and node failures impacting SFCs.
• To optimize VNF migration processes for enhanced network resilience.

Main Methods:

• Developed an offline reinforcement learning-based parallel migration optimization algorithm (RL-PMO).
• Employed a two-stage framework: heuristic algorithms for migration trajectory generation and a Decision Mamba model for policy network training.
• Utilized a twin-critic architecture and CQL regularization to mitigate distribution shift and Q-value overestimation.

Main Results:

• RL-PMO achieved approximately a 95% migration success rate across various load conditions.
• Demonstrated performance improvements of ~13% under low/medium loads and up to 17% under high loads compared to traditional offline RL algorithms.
• Effectively captured dependencies between VNFs and underlying resources using the Decision Mamba model.

Conclusions:

• RL-PMO offers an efficient, reliable, and resource-aware solution for SFC migration during node failures.
• The proposed algorithm enhances the robustness and performance of edge networks.
• Offline reinforcement learning, combined with advanced models like Decision Mamba, shows promise for complex network management tasks.