Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control.

Area of Science:

• Control Systems Engineering
• Wireless Communication Networks
• Stochastic Systems Theory

Background:

• Wireless networked control systems (WNCS) face challenges in joint uplink/downlink scheduling with limited frequency channels.
• Ensuring system stability in WNCS requires considering both control and communication parameters.

Purpose of the Study:

• To develop a stable and optimal transmission scheduling policy for fully distributed WNCS.
• To address the complexity of large action spaces in reinforcement learning for WNCS scheduling.

Main Methods:

• Derivation of a sufficient stability condition for WNCS using stochastic systems theory.
• Formulation of the scheduling problem as a Markov decision process.
• Development of a deep reinforcement learning (DRL) framework with novel action space reduction and embedding techniques.

Main Results:

• A stationary and deterministic scheduling policy is shown to stabilize WNCS when the derived stability condition is met.
• The proposed DRL framework effectively handles large action spaces.
• Numerical results demonstrate superior performance compared to benchmark policies.

Conclusions:

• The DRL-based framework provides an effective solution for joint uplink/downlink scheduling in WNCS.
• The proposed action space management techniques enhance DRL applicability to complex scheduling problems.
• The study establishes a link between system stability conditions and achievable scheduling policies in WNCS.