Adaptive Optimal Control for Stochastic Multiplayer Differential Games Using On-Policy and Off-Policy Reinforcement...

Area of Science:

• Control Theory
• Game Theory
• Machine Learning

Background:

• Traditional differential games often assume deterministic or simply noisy dynamics.
• Realistic systems face complex, multidimensional environmental uncertainties.
• Existing methods struggle with randomly time-varying parameters in multiplayer scenarios.

Purpose of the Study:

• To formulate and solve stochastic multiplayer differential games with general uncertain linear dynamics.
• To address limitations of existing models in handling complex uncertainties.
• To develop online learning algorithms for finding Nash equilibrium solutions.

Main Methods:

• Formulation of two differential games: two-player zero-sum and multiplayer nonzero-sum.
• Derivation of optimal control policies (Nash equilibrium) from Hamiltonian functions.
• Integration of reinforcement learning (RL) and multivariate probabilistic collocation method (MPCM) for online solutions.
• Development of on-policy and off-policy integral reinforcement learning (IRL) algorithms.

Main Results:

• Optimal control policies derived from Hamiltonian functions for uncertain linear dynamics.
• Stability of solutions proven using Lyapunov-type analysis.
• Demonstration that proposed IRL algorithms effectively identify Nash equilibrium solutions for stochastic multiplayer differential games.

Conclusions:

• The proposed framework successfully models and solves stochastic multiplayer differential games with complex uncertainties.
• The integrated RL and MPCM approach enables effective online computation of Nash equilibrium strategies.
• This work advances optimal control solutions for realistic, uncertain multi-agent systems.