Discounted Inverse Reinforcement Learning for Linear Quadratic Control.

Area of Science:

• Control Theory
• Machine Learning
• Reinforcement Learning

Background:

• Linear quadratic control with unknown dynamics and value functions is a significant challenge.
• Existing methods primarily address regulation but struggle with tracking problems.

Purpose of the Study:

• To develop a model-independent method for solving both linear quadratic regulation and tracking problems.
• To address continuous-time systems with unknown value functions and dynamics.

Main Methods:

• A discounted inverse reinforcement learning (DIRL) approach is proposed.
• An error metric is constructed and minimized using a quasi-Newton algorithm to recover the value function and control gain.
• Three DIRL algorithms are presented: model-based, model-free off-policy, and model-free on-policy.

Main Results:

• The proposed DIRL method effectively handles linear quadratic regulation and tracking problems.
• Model-free algorithms utilize expert demonstrations or online data without prior system knowledge.
• Stability, convergence, and existence conditions for solutions are rigorously analyzed.

Conclusions:

• The developed DIRL framework offers a robust solution for linear quadratic control with unknown system properties.
• Numerical simulations validate the theoretical findings and demonstrate the method's effectiveness.