Output Feedback Q-Learning for Linear-Quadratic Discrete-Time Finite-Horizon Control Problems

Area of Science:

• Control Theory
• Systems Engineering
• Optimization

Background:

• Finite-horizon linear-quadratic (LQ) optimal control problems typically require full knowledge of system dynamics.
• Output feedback control is desirable for practical applications where full state information is unavailable.
• Existing methods often necessitate explicit system matrices for controller design.

Purpose of the Study:

• To develop an algorithm for determining output feedback policies for finite-horizon LQ optimal control.
• To enable optimal control without prior knowledge of the system's dynamical matrices.
• To utilize input-output data for controller synthesis.

Main Methods:

• Characterizing Q-factors in the state feedback case for finite-horizon LQ problems.
• Parameterizing Q-factors as functions of input-output vectors.
• Developing a procedure for estimating these Q-factor functions from measured input-output data.

Main Results:

• The proposed algorithm successfully determines output feedback policies for finite-horizon LQ problems.
• The method effectively bypasses the need for explicit system dynamical matrices.
• Optimal control can be computed using estimated Q-factor functions derived from input-output data.

Conclusions:

• The developed algorithm provides a data-driven approach to output feedback control for finite-horizon LQ problems.
• This method enhances the practicality of optimal control in scenarios with limited system knowledge.
• The findings contribute to advancing robust and adaptive control strategies.