Multi-Agent Reinforcement Learning Based Fully Decentralized Dynamic Time Division Configuration for 5G and B5G...

Area of Science:

• Telecommunications Engineering
• Artificial Intelligence
• Network Optimization

Background:

• Future network services demand adaptive uplink and downlink traffic management.
• 5G New Radio (NR) requires dynamic adjustment of time-domain duplex patterns.
• Configuring effective dynamic time division duplex (D-TDD) patterns for 5G NR remains an open research challenge.

Purpose of the Study:

• To propose a decentralized D-TDD configuration method using distributed multi-agent deep reinforcement learning (MARL).
• To maximize the sum rates of all users (UE) by optimizing the D-TDD configuration policy.
• To reduce signaling overhead through a fully decentralized MARL approach.

Main Methods:

• Modeling the D-TDD configuration as a dynamic programming problem and defining the policy as a conditional probability distribution.
• Implementing a decentralized MARL solution where each base station (BS) acts as an agent, using local buffer length observations.
• Integrating a leniency controller and a binary LSTM (BLSTM) based auto-encoder to address MARL's global information limitations and handle complex data.

Main Results:

• The proposed distributed MARL method achieves stable convergence across diverse network environments.
• The system effectively configures uplink and downlink time slot ratios based on local user queue buffer lengths.
• The MARL approach, deployed on Mobile Edge Computing (MEC) servers, demonstrates superior performance compared to traditional distributed deep reinforcement algorithms.

Conclusions:

• The developed distributed MARL framework provides an effective and stable solution for dynamic TDD configuration in 5G networks.
• Decentralized decision-making based on local observations, enhanced by leniency control and auto-encoders, optimizes network resource allocation.
• This approach offers a promising direction for enhancing the adaptability and efficiency of future wireless communication systems.