Memory-Efficient Inverse Reinforcement Learning for Multiplayer Differential Games

Data-driven inverse reinforcement learning (RL) control aims to infer the unknown cost function of a learner system from expert demonstrations. The convergence of existing methods necessitates a data storage mechanism to maintain persistent excitation (PE), which consumes memory and induces delays in satisfying full-rank conditions. To address these problems, in this article, we propose a novel memory-efficient inverse RL algorithm for multiplayer differential game that eliminates the need for strict PE and data storage. We prove that Nash equilibrium solutions for the learner system can be guaranteed under a mild initial excitation condition. Besides, existing inverse RL control algorithms often rely on an initial admissible control policy (IACP), which is difficult to obtain in data-driven scenarios. We address this problem by designing a novel filter-based homotopic RL algorithm, which derives an IACP for learner systems by shifting unstable poles into a stable region. Moreover, we establish several properties of the designed algorithms, including convergence, nonuniqueness, and stability. Finally, the effectiveness of the proposed algorithms is verified by comparative studies and simulation results.