Memory-Efficient Inverse Reinforcement Learning for Multiplayer Differential Games

Area of Science:

• Control Theory
• Machine Learning
• Artificial Intelligence

Background:

• Data-driven inverse reinforcement learning (RL) control infers system dynamics from expert data.
• Existing methods require persistent excitation (PE) and data storage, causing memory and delay issues.

Purpose of the Study:

• To propose a novel, memory-efficient inverse RL algorithm for model-dynamic games (MDG).
• To eliminate the need for strict PE and data storage in RL control.
• To address the challenge of obtaining initial admissible control policies (IACP) in data-driven scenarios.

Main Methods:

• Developed a memory-efficient inverse RL algorithm for MDG, removing strict PE and data storage requirements.
• Proved Nash equilibrium solutions are guaranteed under mild initial excitation.
• Designed a filter-based homotopic RL algorithm to derive IACP by stabilizing system poles.

Main Results:

• The proposed algorithm eliminates memory consumption and delays associated with data storage and PE.
• Guaranteed convergence to Nash equilibrium solutions under a mild initial excitation condition.
• Effectiveness verified through comparative studies and simulations, demonstrating convergence, nonuniqueness, and stability.

Conclusions:

• The novel memory-efficient inverse RL algorithm advances data-driven control for MDG.
• The filter-based homotopic approach provides a viable solution for obtaining IACP.
• The algorithms offer improved efficiency and guaranteed performance in RL control applications.