Data-Driven Robust Control of Discrete-Time Uncertain Linear Systems via Off-Policy Reinforcement Learning

Area of Science:

• Control Theory
• Machine Learning
• Systems Engineering

Background:

• Robust stabilization of discrete-time linear systems is challenging due to bounded and mismatched uncertainties.
• Traditional methods often require precise system models, limiting their applicability.
• The algebraic Riccati equation (ARE) is central to optimal controller design for such systems.

Purpose of the Study:

• To present a model-free solution for the robust stabilization problem of discrete-time linear dynamical systems.
• To develop an optimal controller design method that addresses bounded and mismatched uncertainty.
• To leverage reinforcement learning (RL) for solving the robust control problem without system identification.

Main Methods:

• Derivation of an optimal controller design method involving the solution of an algebraic Riccati equation (ARE).
• Application of off-policy reinforcement learning (RL) to solve the ARE in a model-free manner.
• Comparative analysis of on-policy and off-policy RL methods concerning robustness and system dynamics dependence.

Main Results:

• The optimal controller derived from the ARE successfully achieves robust stabilization of the uncertain system.
• Off-policy RL provides a viable model-free alternative for solving the robust control problem.
• Off-policy RL demonstrates advantages in robustness to probing noise compared to on-policy methods.

Conclusions:

• The proposed off-policy reinforcement learning approach effectively solves the robust stabilization problem for discrete-time linear systems without requiring system models.
• The method validates the efficacy of using RL for complex control tasks with uncertainties.
• This work contributes a novel, data-driven strategy for robust control design.