OpenHoldem: A Benchmark for Large-Scale Imperfect-Information Game Research

Area of Science:

• Computer Science
• Artificial Intelligence
• Game Theory

Background:

• No-limit Texas hold'em (NLTH) is a key testbed for imperfect-information games.
• Progress in superhuman AI for NLTH has been made, but lacks standardized benchmarks.
• This hinders research reproducibility and development for new researchers.

Purpose of the Study:

• To introduce OpenHoldem, an integrated benchmark for large-scale imperfect-information game research.
• To provide a standardized evaluation protocol for NLTH AI.
• To facilitate further studies on theoretical and computational issues in AI for imperfect-information games.

Main Methods:

• Developed a standardized evaluation protocol for NLTH AI.
• Created four publicly available strong baselines for NLTH AI.
• Implemented an online testing platform with APIs for public AI evaluation.

Main Results:

• Established a comprehensive benchmark, OpenHoldem, for NLTH AI research.
• Provided accessible resources including evaluation protocols, baselines, and an online platform.
• Enabled standardized comparison and testing of different NLTH AI methods.

Conclusions:

• OpenHoldem addresses the need for standardized benchmarks in NLTH AI research.
• The platform is expected to accelerate studies on opponent modeling and human-computer interactive learning.
• Public release of OpenHoldem aims to foster further advancements in imperfect-information game AI.