Optimal Output-Feedback Control of Unknown Continuous-Time Linear Systems Using Off-policy Reinforcement Learning.

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Machine Learning

Background:

• Optimal output-feedback control (OPFB) is crucial for linear systems.
• Existing methods often require system model knowledge or are limited in scope.

Purpose of the Study:

• To develop a model-free, off-policy reinforcement learning algorithm for optimal OPFB control.
• To unify the design of OPFB controllers for both regulation and tracking problems.

Main Methods:

• A discounted performance function and a discounted algebraic Riccati equation (ARE) were employed.
• System state reconstruction from limited output observations was demonstrated.
• A Bellman equation was developed for policy evaluation and improvement using limited output data.

Main Results:

• A model-free, off-policy RL-based OPFB controller was successfully developed.
• The method does not require knowledge of system state or dynamics.
• The proposed OPFB method demonstrated superior performance, equivalent to state-feedback control.

Conclusions:

• The developed algorithm provides a powerful, unified framework for optimal OPFB control.
• It effectively addresses both regulation and tracking problems in linear systems.
• This approach advances model-free control design using reinforcement learning.