Model-Based Reinforcement Learning with Automated Planning for Network Management.

Area of Science:

• Artificial Intelligence
• Network Management
• Machine Learning

Background:

• Reinforcement Learning (RL) offers automation potential for network management but its trial-and-error nature limits model-based RL (MBRL) applications.
• Existing MBRL methods face challenges in complex network environments requiring explainable predictions or resource constraints.

Purpose of the Study:

• To investigate the integration of Automated Planning (AP) with RL for network management.
• To compare different AP-RL integration strategies and propose a cognitive management control loop architecture.
• To evaluate the performance of AP-based MBRL in simulated network management scenarios.

Main Methods:

• Exploration of AP techniques to enable MBRL in network management.
• Development and comparison of various AP and RL integration strategies.
• Implementation of a cognitive management control loop architecture combining AP and RL.
• Experimental evaluation in a simulated network environment.

Main Results:

• The proposed AP-RL combination improves upon model-free RL.
• Performance in terms of reward and convergence time is lower compared to Deep Reinforcement Learning (Deep RL).
• AP-based MBRL provides a viable alternative when model interpretability is crucial or Deep RL's computational demands are prohibitive.

Conclusions:

• Integrating Automated Planning with Reinforcement Learning presents a promising approach for network management automation.
• AP-based MBRL offers distinct advantages in scenarios demanding model transparency and computational efficiency.
• This hybrid approach enhances traditional RL methods and provides a valuable tool for specific network management challenges.