Neural Q-learning for discrete-time nonlinear zero-sum games with adjustable convergence rate.

Area of Science:

• Control Theory
• Artificial Intelligence
• Game Theory

Background:

• Zero-sum games present complex control challenges.
• Model-free approaches are needed for practical applications.
• Accelerating iterative learning algorithms is crucial for efficiency.

Purpose of the Study:

• To develop an adjustable Q-learning scheme for discrete-time nonlinear zero-sum games.
• To enhance the convergence rate of Q-function iteration.
• To enable model-free tracking control using neural networks.

Main Methods:

• Analysis of monotonicity and convergence for iterative Q-functions.
• Integration of neural networks for model-free control.
• Design of two accelerated Q-learning algorithms with convergence guarantees.

Main Results:

• An adjustable Q-learning scheme accelerates convergence.
• Two algorithms ensure convergence with adaptive acceleration phases or adjustable relaxation factors.
• Simulations confirm the algorithm's effectiveness in a practical scenario.

Conclusions:

• The proposed adjustable Q-learning scheme effectively solves nonlinear zero-sum games.
• Neural network integration facilitates model-free tracking control.
• The developed algorithms offer accelerated and guaranteed convergence for Q-learning.