Two-Stage Asynchronous Learning for Optimal Tracking in Multiplayer Differential Games.

Area of Science:

• Control Theory
• Game Theory
• Machine Learning

Background:

• Multiplayer differential game systems (MDGS) present complex control challenges, especially with unknown system dynamics.
• Achieving Nash equilibrium solutions is crucial for optimal decentralized control in such systems.
• Existing methods often require initial admissible control policies, limiting their applicability.

Purpose of the Study:

• To investigate the optimal tracking control problem for MDGS with unknown dynamics.
• To develop a learning scheme that achieves Nash equilibrium solutions without prior control policy knowledge.
• To enhance convergence efficiency using asynchronous policy updates.

Main Methods:

• A two-stage asynchronous learning scheme is proposed.
• Stage 1: Stabilizing control policies via a homotopic-based iterative process.
• Stage 2: Asynchronous policy iteration (PI) using partial real-time information, extended to a data-driven framework.

Main Results:

• The proposed scheme successfully achieves Nash equilibrium solutions for optimal tracking control.
• The asynchronous PI method demonstrates improved convergence efficiency over synchronous approaches.
• Theoretical convergence is proven under stabilizability and detectability conditions.
• Simulation examples confirm the method's effectiveness in tracking sinusoidal references.

Conclusions:

• The developed two-stage asynchronous learning scheme effectively solves the optimal tracking control problem for MDGS with unknown dynamics.
• The data-driven extension removes the need for explicit system dynamic information.
• The proposed algorithm shows superiority compared to existing methods, validated through simulations.