A unified and efficient training framework for open-ended non-transitive games.

Area of Science:

• Game Theory
• Artificial Intelligence
• Reinforcement Learning

Background:

• Self-play (SP) and Policy-Space Response Oracles (PSRO) are key frameworks for finding Nash equilibrium (NE) in games.
• SP excels in transitive games but struggles with non-transitive ones.
• PSRO handles non-transitive games but is inefficient due to retraining policies from scratch.

Purpose of the Study:

• To develop a novel framework, Unified and Efficient Play (UEP), that merges the strengths of SP and PSRO.
• To enable efficient NE solutions in open-ended, non-transitive games with warm-start capabilities.
• To enhance training efficiency by balancing policy accuracy and diversity.

Main Methods:

• Introduction of the Unified and Efficient Play (UEP) framework.
• Development of a unified regularized distance metric to balance SP accuracy and PSRO diversity.
• Theoretical analysis for UEP convergence to NE.
• Empirical validation in diverse open-ended, non-transitive games.

Main Results:

• UEP framework effectively combines SP and PSRO advantages.
• The proposed regularized distance metric improves training efficiency.
• Theoretical convergence of UEP to NE is established.
• Empirical results demonstrate UEP's superior performance in approximating NE and generating robust policies.

Conclusions:

• UEP offers a significant advancement for solving NE in challenging game environments.
• The framework provides a more efficient and robust alternative to existing SP and PSRO methods.
• UEP demonstrates strong potential for applications in complex, open-ended games.