Related Experiment Video
Updated: Mar 29, 2026
08:18
WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control
Published on: August 15, 2020
5.5K
Online Solution of Two-Player Zero-Sum Games for Continuous-Time Nonlinear Systems With Completely Unknown Dynamics
IEEE Transactions on Neural Networks and Learning Systems
|November 25, 2015
Summary
This study introduces an online adaptive algorithm for learning optimal policies in continuous-time nonlinear games with unknown dynamics. It effectively finds Nash equilibrium solutions without prior system knowledge.
Area of Science:
- Control Theory
- Game Theory
- Adaptive Systems
Background:
- Two-player zero-sum games with continuous-time nonlinear systems present challenges due to unknown dynamics.
- Finding Nash equilibrium solutions (optimal policy pairs) is crucial for system control and decision-making.
Purpose of the Study:
- To develop an online adaptive algorithm for learning Nash equilibrium solutions in continuous-time nonlinear zero-sum games with completely unknown dynamics.
- To address the limitations of existing methods that require prior knowledge of system dynamics.
Main Methods:
- Review of the Simultaneous Policy Updating Algorithm (SPUA) for known systems with a novel convergence proof.
- Development of an online algorithm based on SPUA that learns the Hamilton-Jacobi-Isaacs (HJI) equation solution or generalized algebraic Riccati equation solution in real-time.
- Utilizing the recursive least square method for simultaneous identification of unknown system parameters.
Main Results:
- The proposed online algorithm simultaneously learns the optimal policy pair and the solution to the HJI equation (or generalized algebraic Riccati equation for linear systems).
- Convergence of the online algorithm to the optimal solutions is analytically proven.
- A practical online algorithm is developed and demonstrated through simulations.
Conclusions:
- The developed online adaptive algorithm effectively learns Nash equilibrium solutions for continuous-time nonlinear zero-sum games with unknown dynamics.
- The method provides a robust approach for real-time policy optimization without requiring a priori system information.
- Simulation results validate the practical applicability and effectiveness of the proposed algorithm.
Related Concept Videos
01:13Parameters Affecting Nonlinear Elimination: Zero-Order Input, First-Order Absorption and Two-Compartment Model
391
Drugs administered through various routes can lead to nonlinear elimination, resulting in complex pharmacokinetic behaviors crucial to understanding efficacious drug dosing.
When a drug is administered through a constant intravenous infusion and eliminated via nonlinear pharmacokinetics, it follows zero-order input. For example, oral drugs undergo first-order absorption upon administration and are eliminated through nonlinear pharmacokinetics.
In the case of subcutaneously administered drugs,...
When a drug is administered through a constant intravenous infusion and eliminated via nonlinear pharmacokinetics, it follows zero-order input. For example, oral drugs undergo first-order absorption upon administration and are eliminated through nonlinear pharmacokinetics.
In the case of subcutaneously administered drugs,...
391
01:21Linear Approximation in Time Domain
406
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
406
01:21Differential Equations: Problem Solving
173
When analyzing the motion of falling objects, it is essential to consider not only the force of gravity but also the opposing force of air resistance. A practical example involves releasing a heavy test weight during a safety check on a ship. As the weight falls from rest, gravity accelerates it downward while air resistance exerts an upward force that increases with velocity. This dynamic interplay of forces is well described by differential equations, which provide a mathematical framework...
173
01:30Gaussian Elimination: Problem Solving
277
Systems of linear equations in several variables are pivotal in modeling complex scenarios involving multiple unknowns and constraints. Such systems are widely used in various fields to represent relationships where several conditions must be simultaneously satisfied. Each variable in the system corresponds to an unknown quantity, while each equation imposes a linear constraint, leading to a structured approach for analyzing and solving real-world problems.A system of three equations with three...
277
01:27Linear Differential Equations
206
The integrating factor method provides a systematic way to solve first-order linear differential equations, especially those that cannot be handled by separation of variables. This method is particularly useful in modeling time-dependent physical systems influenced by both constant inputs and resistive forces. A common example is the motion of a car subjected to a constant engine force while experiencing air resistance proportional to its velocity.In such scenarios, Newton’s second law...
206
01:31Alternative Sets of Equilibrium Equations
1.1K
When analyzing the behavior of structures, engineers often rely on the concept of equilibrium. This refers to the state where all forces and moments acting on a system balance each other, resulting in no net movement or rotation. In many cases, equilibrium can be described by a set of standard equations. However, in some situations, alternative sets of equilibrium equations must be used to describe the system's behavior accurately.
One example of such a situation can be observed in a...
One example of such a situation can be observed in a...
1.1K