Bias-compensated Q-learning for optimal tracking control under denial-of-service attacks.

Area of Science:

• Control Systems Engineering
• Machine Learning
• Cybersecurity

Background:

• Networked control systems (NCS) are crucial in modern applications.
• Learning optimal control policies from data under communication attacks, like denial-of-service (DoS), is challenging.
• Existing methods struggle with data efficiency and robustness in adversarial environments.

Purpose of the Study:

• To develop a data-driven optimal tracking control method for NCS under DoS attacks.
• To address challenges posed by control-input losses and enable robust learning.
• To achieve high data efficiency and computational simplicity in a model-free framework.

Main Methods:

• Propose a bias-compensated Q-learning algorithm for knowledge-informed optimal tracking control.
• Design a bias compensation mechanism to handle control-input losses caused by DoS attacks.
• Utilize a model-free framework that requires minimal system dynamics knowledge.

Main Results:

• The proposed method enables direct data-driven learning under DoS attacks without explicit expectation computation.
• Numerical simulations demonstrate learning a Riccati-equivalent optimal policy from data under 40% DoS attacks on a quadrotor system.
• The approach maintains high data efficiency, computational simplicity, and robustness.

Conclusions:

• The bias-compensated Q-learning algorithm effectively learns optimal control policies in NCS despite DoS attacks.
• This model-free approach offers a robust and efficient solution for secure networked control.
• The findings have significant implications for the security and reliability of autonomous systems.