Nash Equilibrium in Multiplayer Graphical Games via Reinforcement Learning and Distributed Observers.

Area of Science:

• Game Theory
• Reinforcement Learning
• Control Systems

Background:

• Existing multiplayer game theory research often assumes fully connected networks.
• Multiplayer graphical games offer more practical, sparser communication topologies for large systems.
• External system influences pose challenges in computing equilibrium strategies.

Purpose of the Study:

• To compute Nash equilibrium (NE) strategies in multiplayer graphical games influenced by an external system.
• To develop a distributed adaptive observer for estimating unknown external system states.
• To ensure system stability and enhance convergence speed for NE strategies.

Main Methods:

• Utilizing a reinforcement learning (RL) approach.
• Proposing a distributed adaptive observer to estimate external system states, proving asymptotic convergence of observation error.
• Deriving discount factor ranges for system stability.
• Developing an off-policy algorithm integrated with the observer for policy evaluation.
• Implementing a distributed policy improvement mechanism for faster convergence.

Main Results:

• The distributed adaptive observer achieves zero asymptotic observation error.
• Specific discount factor ranges are identified to maintain system stability.
• The integrated off-policy algorithm and distributed policy improvement mechanism ensure policy convergence to NE.
• The proposed method maintains system stability throughout the process.
• Simulations on a voltage synchronization system validate the algorithm's effectiveness.

Conclusions:

• The proposed RL-based method effectively computes Nash equilibrium strategies in multiplayer graphical games with external influences.
• The integration of a distributed adaptive observer and policy improvement mechanism enhances both accuracy and convergence speed.
• The approach guarantees system stability, making it suitable for practical large-scale applications.