Mastering Atari, Go, chess and shogi by planning with a learned model.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Planning capabilities are crucial for artificial intelligence agents.
• Tree-based planning excels in domains with perfect simulators (e.g., chess, Go).
• Real-world problems often involve complex, unknown environmental dynamics, hindering traditional planning methods.

Purpose of the Study:

• To introduce the MuZero algorithm, a novel approach for artificial intelligence planning.
• To demonstrate MuZero's ability to achieve high performance without prior knowledge of environmental dynamics.
• To evaluate MuZero's effectiveness across diverse and visually complex domains.

Main Methods:

• MuZero combines tree-based search with a learned model.
• The algorithm learns an iterable model that predicts policy, value, and reward.
• This learned model supports planning in unknown environments.

Main Results:

• MuZero achieved state-of-the-art performance on 57 Atari games, a domain where model-based planning historically struggled.
• MuZero matched the superhuman performance of AlphaZero in Go, chess, and shogi without knowledge of game rules.
• The algorithm demonstrates strong planning capabilities in visually complex and dynamic environments.

Conclusions:

• The MuZero algorithm represents a significant advancement in artificial intelligence planning.
• MuZero's learned model approach overcomes limitations of model-based planning in unknown environments.
• This method offers a powerful new tool for developing intelligent agents capable of tackling complex real-world problems.