Inverse Reinforcement Learning in Tracking Control Based on Inverse Optimal Control.

Area of Science:

• Control Systems Engineering
• Machine Learning
• Robotics

Background:

• Reinforcement learning (RL) is crucial for control systems.
• Learning unknown performance objectives is a key challenge.
• Existing methods lack clarity on the relationship between inverse RL and IOC.

Purpose of the Study:

• To develop a novel inverse reinforcement learning (RL) algorithm for learning unknown performance objective functions in tracking control.
• To clarify the relationship between inverse RL and inverse optimal control (IOC).
• To characterize the non-uniqueness of reward weights for control policies.

Main Methods:

• A three-step algorithm combining optimal control update, gradient descent correction, and inverse optimal control (IOC) update.
• Development of a model-based algorithm.
• Development of two model-free algorithms for systems with unknown dynamics.

Main Results:

• The proposed algorithm effectively learns unknown performance objectives.
• The study clarifies the relationship between inverse RL and IOC.
• Characterization of the set of all reward weights generating a target control policy.
• Simulation experiments validate the effectiveness of the developed algorithms.

Conclusions:

• The novel inverse RL algorithm successfully identifies unknown performance objectives for tracking control.
• The research provides a unified framework for understanding inverse RL and IOC.
• The developed model-based and model-free algorithms offer practical solutions for systems with incomplete information.