Cooperative Finitely Excited Learning for Dynamical Games.

Area of Science:

• Control Theory
• Game Theory
• Machine Learning

Background:

• Conventional centralized actor-critic learning struggles with continuous-time zero-sum games.
• Existing methods often require strict persistent excitation conditions.

Purpose of the Study:

• To enhance the learning framework for continuous-time zero-sum games.
• To develop a novel cooperative learning approach for improved efficiency and stability.

Main Methods:

• A cooperative finitely excited learning approach combining online and instantaneous data.
• Utilizing experience replay and distributed agent interaction.
• Replacing persistent excitation with a cooperative excitation condition.

Main Results:

• Guaranteed consensus of distributed actor-critic learning on the Hamilton-Jacobi-Isaacs (HJI) equation solution.
• Ensured closed-loop stability of the equilibrium point.
• Guaranteed convergence to the Nash equilibrium.

Conclusions:

• The proposed cooperative learning method enhances efficiency and stability in continuous-time zero-sum games.
• The approach simplifies excitation conditions while ensuring convergence and stability.
• Simulation results validate the method's efficacy over prior approaches.