Inverse reinforcement learning for discrete-time linear systems based on inverse optimal control.

Area of Science:

• Control Systems Engineering
• Machine Learning
• Optimization

Background:

• Optimal control problems require accurate cost functions for effective system design.
• Inverse reinforcement learning (IRL) aims to infer these cost functions from expert demonstrations.
• Linear time-invariant systems are fundamental in many engineering applications.

Purpose of the Study:

• To develop novel IRL algorithms for reconstructing cost functions in discrete-time linear time-invariant systems.
• To address both model-based and partially model-free scenarios for cost function recovery.
• To ensure the stability and convergence of the proposed IRL methods.

Main Methods:

• A model-based IRL algorithm is presented by reformulating the optimal control gain formula.
• A partially model-free IRL framework is developed using auxiliary control inputs and an outer-inner loop structure.
• Algorithms involve updating control gain via algebraic Riccati equation (ARE), gradient descent for cost matrix correction, and weight matrix updates via inverse optimal control (IOC).

Main Results:

• The proposed algorithms successfully reconstruct the cost function using expert agent data (state and input measurements).
• The partially model-free approach enables cost function reconstruction even when the input matrix is unknown.
• Convergence of the algorithms and stability of the closed-loop system are theoretically demonstrated.

Conclusions:

• The developed IRL algorithms are effective for cost function reconstruction in linear systems.
• The methods provide robust solutions for both known and partially unknown system models.
• Simulation results validate the practical applicability and performance of the proposed IRL techniques.