Data-Driven Inverse Reinforcement Learning Control for Linear Multiplayer Games.

Area of Science:

• Control Theory
• Game Theory
• Machine Learning

Background:

• Multilayer games in linear systems present complex control challenges.
• Inverse reinforcement learning (RL) aims to infer cost functions from expert behavior.
• Existing methods often require knowledge of system dynamics or expert gains.

Purpose of the Study:

• To propose a data-driven inverse RL control algorithm for nonzero-sum multiplayer games.
• To develop a method for reconstructing unknown expert cost functions using demonstrated trajectories.
• To enable control policy acquisition without prior knowledge of system dynamics.

Main Methods:

• A model-based inverse RL policy iteration framework is proposed, involving policy evaluation, state-reward weight improvement, and policy improvement.
• An online, data-driven, off-policy inverse RL algorithm is developed based on the model-based approach.
• Lyapunov functions and inverse optimal control (IOC) are utilized for cost function reconstruction.

Main Results:

• The proposed algorithms allow a learner to obtain expert control feedback gains and trajectories using only system trajectory data.
• The online off-policy algorithm guarantees unbiased solutions, even with added probing noise satisfying the persistence of excitation (PE) condition.
• Convergence and stability analyses confirm the theoretical soundness of the developed algorithms.

Conclusions:

• The data-driven inverse RL approach effectively solves control problems in multiplayer games without system dynamics knowledge.
• The developed algorithms provide a robust and stable method for learning control policies from expert demonstrations.
• Simulation examples validate the practical effectiveness of the proposed inverse RL techniques.