COPSRO: An Offline Empirical Game Theoretic Method With Conservative Critic.

Area of Science:

• Artificial Intelligence
• Game Theory
• Machine Learning

Background:

• Empirical Game-Theoretic Analysis (EGTA) typically requires extensive environmental interactions to model complex multiagent systems.
• Existing methods suffer from low data utilization efficiency and are often infeasible for risk-sensitive applications.

Purpose of the Study:

• To develop a novel algorithm for learning approximate Nash equilibrium (NE) from static, historical datasets in an offline setting.
• To address the limitations of active data collection in EGTA, particularly for risk-averse scenarios.

Main Methods:

• Introduced the Conservative Offline Policy Space Response Oracle (COPSRO) algorithm for identifying NE from fixed datasets.
• Constructed an overcomplete strategy population from offline data to approximate the game's policy space.
• Integrated a conservative critic (CC) to mitigate overestimation issues common in offline learning.
• Developed an offline NE solver for iterative computation of approximate NE.

Main Results:

• COPSRO successfully identifies equilibrium strategies without requiring real-world interaction.
• The algorithm demonstrates superior convergence and exploitability compared to existing methods, especially with low dataset coverage (10-20%).
• Empirical evaluations validate the effectiveness of COPSRO across various real-world tasks.

Conclusions:

• COPSRO offers a viable and efficient approach for learning NE in offline settings, enhancing applicability in risk-averse domains.
• The method significantly improves data utilization and performance over traditional EGTA approaches when dealing with limited historical data.