An Edge Server Placement Method Based on Reinforcement Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Network Engineering

Background:

• Mobile edge computing (MEC) server placement is a complex multi-objective optimization problem.
• Existing methods like mixed integer programming and heuristic algorithms suffer from scalability issues, local optima, and tuning difficulties.

Purpose of the Study:

• To propose a novel edge server placement algorithm, DQN-ESPA, based on deep Q-network and reinforcement learning.
• To achieve optimal edge server placements without prior experience, overcoming limitations of traditional approaches.

Main Methods:

• Modeling the edge server placement problem as a Markov decision process (MDP).
• Formalizing the MDP with state space, action space, and reward function.
• Solving the MDP using a reinforcement learning algorithm (deep Q-network).

Main Results:

• DQN-ESPA demonstrated superior performance compared to Simulated Annealing Placement Algorithm (SAPA), Top-K Placement Algorithm (TKPA), K-Means Placement Algorithm (KMPA), and Random Placement Algorithm (RPA).
• Experimental results on Shanghai Telecom datasets showed significant improvements in placement performance.
• Achieved up to 13.40% and 15.54% better placement for 100 and 300 edge servers, respectively, by considering access delay and workload balance.

Conclusions:

• DQN-ESPA offers an effective and scalable solution for the edge server placement problem in MEC systems.
• The reinforcement learning approach enables optimal placements by learning from the environment without relying on previous data or experience.
• The algorithm provides significant performance gains, particularly in scenarios demanding low access delay and balanced workloads.