Adaptive Dynamic Programming for Discrete-Time Zero-Sum Games.

Area of Science:

• Control Theory
• Game Theory
• Optimization

Background:

• Solving infinite-horizon discrete-time two-player zero-sum games for nonlinear systems is computationally challenging.
• Existing methods often require specific conditions, such as the existence of a saddle-point equilibrium, limiting their applicability.

Purpose of the Study:

• To develop a novel adaptive dynamic programming (ADP) algorithm for solving infinite-horizon discrete-time two-player zero-sum games of nonlinear systems.
• To provide a convergence analysis that guarantees convergence to optimal solutions, irrespective of saddle-point equilibrium existence.

Main Methods:

• Development of an iterative zero-sum ADP algorithm.
• Initialization of upper and lower iterations using arbitrary positive semidefinite functions.
• Novel convergence analysis to ensure convergence of value functions to optimums.

Main Results:

• The algorithm converges to optimal solutions for zero-sum games when a saddle-point equilibrium exists, without requiring its explicit existence criteria.
• When a saddle-point equilibrium does not exist, the algorithm yields upper and lower optimal performance index functions.
• The upper and lower performance index functions are proven to be non-equivalent in cases without saddle-point equilibrium.

Conclusions:

• The proposed iterative zero-sum ADP algorithm effectively solves nonlinear zero-sum games.
• The algorithm demonstrates robustness by converging to optimal solutions even when saddle-point equilibria are not guaranteed.
• Simulation results validate the algorithm's performance and highlight its advantages over existing methods.