Data-driven inverse optimal control for continuous-time nonlinear systems.

Area of Science:

• Control Theory
• Machine Learning
• Robotics

Background:

• Estimating cost functions is crucial for understanding and replicating expert behavior in autonomous systems.
• Existing inverse reinforcement learning methods often require system models or have high computational costs.

Purpose of the Study:

• To introduce a novel model-free and partially model-free algorithm for inverse optimal control.
• To estimate the cost function of continuous-time nonlinear deterministic systems using expert trajectories.

Main Methods:

• The algorithm utilizes expert input-state trajectories.
• It separately employs control policy information and the Hamilton-Jacobi-Bellman equation for parameter estimation.
• A single forward optimal control problem solve is needed for initialization in the model-free version.

Main Results:

• The proposed algorithm effectively estimates cost function parameters for continuous-time nonlinear deterministic systems.
• The model-free approach reduces computational complexity compared to existing methods.
• The partially model-free version offers further efficiency for systems with known input dynamics.

Conclusions:

• The developed algorithm provides a broadly applicable and computationally efficient solution for inverse reinforcement learning.
• Its effectiveness is validated through simulations, indicating potential for real-world applications in autonomous systems and robotics.