Multiplayer Differential Games of Markov Jump Systems via Reinforcement Learning

Area of Science:

• Control Theory
• Game Theory
• Machine Learning

Background:

• Online multiplayer differential games (MDGs) of Markov jump systems (MJSs) present significant control challenges.
• Existing reinforcement learning (RL) algorithms often require complete system knowledge or synchronous updates.

Purpose of the Study:

• To develop novel RL-based strategies for solving MDGs of MJSs.
• To address limitations of existing RL algorithms in distributed and asynchronous settings.

Main Methods:

• Propose a distributed minmax strategy using distributed game algebraic Riccati equations (DGAREs) and a novel online distributed RL algorithm.
• Develop an online asynchronous RL algorithm for Nash strategy application in MDGs.
• Rigorously analyze the convergence of the proposed RL algorithms.

Main Results:

• The distributed minmax strategy allows players to derive optimal policies without prior knowledge of others' policies.
• The asynchronous RL algorithm incorporates the latest information for policy evaluation and improvement.
• The proposed methods demonstrate effectiveness through simulations on inverted pendulum systems.

Conclusions:

• Novel RL algorithms effectively solve MDGs of MJSs under both distributed minmax and Nash strategies.
• The developed methods offer advancements in decentralized control and adaptive policy optimization for dynamic systems.