Data-Based Reinforcement Learning for Nonzero-Sum Games With Unknown Drift Dynamics.

Area of Science:

• Game Theory
• Control Theory
• Machine Learning

Background:

• Nonlinear optimization problems in nonzero-sum (NZS) games are challenging, especially with unknown system dynamics.
• Approximating Nash equilibrium is crucial for understanding and solving such games.

Purpose of the Study:

• To propose a data-based integral reinforcement learning (IRL) method for solving nonlinear NZS games with unknown drift dynamics.
• To ensure the convergence and practical implementation of the proposed IRL algorithm.

Main Methods:

• Development of a data-based integral reinforcement learning (IRL) algorithm to iteratively approximate the Nash equilibrium.
• Equivalence proof between the data-based IRL method and model-based policy iteration for guaranteed convergence.
• Design of a single-critic neural network structure and critic weight updating laws using offline and online iterative learning, incorporating experience replay.

Main Results:

• The data-based IRL method is proven to be equivalent to model-based policy iteration, ensuring convergence.
• The proposed method utilizes a single-critic neural network and adaptive weight updating laws.
• Experience replay enhances the convergence rate of critic weights, with uniform ultimate boundedness guaranteed by Lyapunov stability analysis.

Conclusions:

• The data-based IRL algorithm effectively solves nonlinear NZS games with unknown drift dynamics.
• The method offers a practical approach for finding Nash equilibria in complex game scenarios.