Learning to play against any mixture of opponents

Area of Science:

• Artificial Intelligence
• Reinforcement Learning
• Game Theory

Background:

• Adapting AI agents to diverse opponent strategies is challenging.
• Retraining agents for new opponent mixtures is computationally expensive.
• Transfer learning offers a potential solution for efficient adaptation.

Purpose of the Study:

• To propose and validate Q-Mixing, a novel transfer learning method.
• To enable agents to construct policies against new opponent mixtures without retraining.
• To enhance agent performance by incorporating opponent policy classification.

Main Methods:

• Q-Mixing learns Q-values against pure-strategy opponents.
• It approximates Q-values for mixed strategies by averaging pure-strategy Q-values.
• An opponent policy classifier refines Q-value mixing using observed data.

Main Results:

• Q-Mixing successfully transfers knowledge across different opponent strategy mixtures.
• The method was validated in grid-world soccer and a social dilemma game.
• Augmenting Q-Mixing with a classifier improved performance over direct training.

Conclusions:

• Q-Mixing provides an effective approach for transfer learning in multi-agent systems.
• The method allows agents to generalize policies to unseen opponent strategy distributions.
• Integrating opponent policy classification further enhances adaptive capabilities.