Nearly Optimal Control for Mixed Zero-Sum Game Based on Off-Policy Integral Reinforcement Learning

Area of Science:

• Control Theory
• Game Theory
• Machine Learning

Background:

• Solving mixed zero-sum games with unknown nonlinear system dynamics presents significant challenges.
• Traditional methods often require complete system information, limiting their applicability.

Purpose of the Study:

• To develop a novel policy iterative algorithm using integral reinforcement learning (IRL) for mixed zero-sum games.
• To achieve optimal control for competing and collaborating players in systems with unknown dynamics.

Main Methods:

• A policy iterative algorithm employing integral reinforcement learning (IRL) is proposed, which bypasses the need for system information.
• An adaptive update law integrating a critic-actor structure with experience replay is introduced.
• Actor functions are designed to approximate optimal control and estimate auxiliary control simultaneously.

Main Results:

• The proposed algorithm successfully obtains optimal control strategies for all players.
• Parameters of the actor-critic structure are updated simultaneously, ensuring efficient learning.
• Uniform ultimate boundedness of parameter errors in polynomial approximation is mathematically proven.

Conclusions:

• The developed IRL-based algorithm effectively solves mixed zero-sum games with unknown nonlinear dynamics.
• The adaptive critic-actor structure with experience replay offers a robust approach to control optimization.
• Simulation results validate the algorithm's effectiveness and practical applicability.