Packet Flow Capacity Autonomous Operation Based on Reinforcement Learning.

Area of Science:

• Computer Science
• Telecommunications Engineering
• Network Management

Background:

• Increasing traffic dynamicity necessitates self-network operation for cost savings.
• Dynamic capacity management of large packet flows is crucial, especially in packet over optical networks.
• Reinforcement Learning (RL) offers autonomicity but faces performance challenges during initial learning and traffic variations.

Purpose of the Study:

• To propose a novel management lifecycle for dynamic capacity management in packet networks.
• To overcome the performance limitations of traditional RL methods in network capacity management.
• To ensure consistent and robust performance from the outset of network operation.

Main Methods:

• A three-phase management lifecycle: 1. Self-tuned threshold-based approach, 2. RL with generic traffic profiles, 3. RL with real traffic profiles.
• Simulation-based evaluation comparing the proposed lifecycle against benchmarking approaches.
• Analysis of performance metrics including delay, packet loss, and capacity overprovisioning.

Main Results:

• RL algorithms exhibit poor performance until the optimal policy is learned and when traffic characteristics change.
• The proposed three-phase lifecycle demonstrates superior performance compared to existing methods.
• The lifecycle achieves noticeable performance improvements from the start and maintains robustness against traffic fluctuations.

Conclusions:

• The proposed management lifecycle effectively addresses the limitations of RL in dynamic network capacity management.
• This approach ensures immediate, high performance and resilience to changing traffic conditions.
• The lifecycle is suitable for deployment in real-world operator networks demanding reliable self-operation.