AlphaZe∗∗: AlphaZero-like baselines for imperfect information games are surprisingly strong

Area of Science:

• Artificial Intelligence
• Game Theory
• Reinforcement Learning

Background:

• Deep neural networks excel in perfect information games using frameworks like AlphaZero.
• Imperfect information games, with their inherent uncertainty, have been considered unsuitable for these methods.
• Current approaches for imperfect information games often rely on heuristics or specialized techniques.

Purpose of the Study:

• To challenge the notion that deep reinforcement learning is unsuitable for imperfect information games.
• To introduce a novel AlphaZero-based framework, AlphaZe**, for imperfect information strategy games.
• To evaluate the learning convergence and performance of AlphaZe** in complex games.

Main Methods:

• Developed AlphaZe**, a novel algorithm based solely on reinforcement learning and inspired by AlphaZero.
• Applied AlphaZe** to strategy games with imperfect information, specifically Stratego and DarkHex.
• Utilized a model-based approach within the reinforcement learning framework.

Main Results:

• AlphaZe** demonstrates surprisingly strong baseline performance in imperfect information games.
• Achieved comparable win rates against established Stratego bots like Pipeline Policy Space Response Oracle (P2SRO).
• Showed significant adaptability to rule changes, outperforming heuristic and oracle-based methods.

Conclusions:

• Deep reinforcement learning frameworks can be effectively adapted for games with imperfect information.
• AlphaZe** presents a viable and robust alternative to traditional methods in this domain.
• The adaptability of AlphaZe** to changing game rules highlights its potential for complex strategic environments.