Output-Feedback Control of Linear Continuous-Time Systems Using Discounted Inverse Reinforcement Learning.

Area of Science:

• Control Systems Engineering
• Machine Learning
• Robotics

Background:

• Discounted inverse reinforcement learning (DIRL) typically requires full-state feedback, limiting its use in real-world applications with only input-output data.
• Unknown continuous-time (CT) systems with partially observable states present significant control challenges.
• Learning unknown discounted value functions is crucial for optimal control policy derivation.

Purpose of the Study:

• To develop a novel model-free, output-feedback (OPFB) DIRL algorithm for linear quadratic (LQ) control of unknown CT systems.
• To address the limitations of existing DIRL methods by enabling learning from input-output data.
• To reconstruct system states using expert control output data for policy learning.

Main Methods:

• A state reconstruction method is designed utilizing expert control and measured output data.
• A model-free OPFB DIRL algorithm is presented to iteratively learn the unknown value function and optimal control policy.
• Rigorous analysis of algorithm convergence and solution uniqueness is performed.

Main Results:

• The proposed algorithm effectively recovers the expert control policy.
• Simulations demonstrate superior computational efficiency compared to state-of-the-art methods.
• The algorithm successfully handles partially observable states and unknown value functions.

Conclusions:

• The novel OPFB DIRL algorithm provides an effective solution for controlling unknown CT systems with limited state information.
• The method enhances the applicability of DIRL in practical scenarios by utilizing only input-output data.
• The algorithm offers a computationally efficient and robust approach to learning optimal control policies.