Multiplayer Stackelberg-Nash Game for Nonlinear System via Value Iteration-Based Integral Reinforcement Learning.

Area of Science:

• Game Theory
• Control Theory
• Reinforcement Learning

Background:

• Multiplayer Stackelberg-Nash games (SNG) involve hierarchical decision-making with one leader and multiple followers.
• Analyzing nonlinear dynamical systems in SNGs presents computational challenges for finding equilibrium points.

Purpose of the Study:

• To develop a novel algorithm for solving multiplayer SNGs in nonlinear dynamical systems.
• To address the analytical difficulties in calculating equilibrium strategies.
• To ensure the admissibility and convergence of the proposed solution.

Main Methods:

• A two-level value iteration-based integral reinforcement learning (VI-IRL) algorithm was developed.
• The algorithm utilizes partial information of system dynamics and neural networks for function approximation.
• Least-squares methods are employed for updating weights.

Main Results:

• The VI-IRL algorithm converges asymptotically to the Stackelberg-Nash equilibrium strategies under weak coupling conditions.
• Effective termination criteria were introduced to guarantee policy admissibility after finite iterations.
• The algorithm's effectiveness was validated through two simulation examples.

Conclusions:

• The proposed VI-IRL algorithm offers an effective method for solving complex multiplayer SNGs in nonlinear systems.
• The approach overcomes analytical limitations by using reinforcement learning and partial system information.
• The method provides a computationally feasible way to determine equilibrium strategies.