Two-stage training algorithm for AI robot soccer.

Area of Science:

• Artificial Intelligence
• Multi-Agent Systems
• Reinforcement Learning

Background:

• Cooperative behavior is crucial in multi-agent reinforcement learning, especially with heterogeneous agents.
• Centralized training with joint-action sets struggles to optimize performance for diverse agent types.
• Existing methods face limitations in achieving effective cooperation among heterogeneous agents.

Purpose of the Study:

• To enhance cooperative learning in heterogeneous multi-agent reinforcement learning (HMARL).
• To improve the performance of heterogeneous agents during centralized training.
• To develop a method that maximizes both individual role rewards and collective team rewards.

Main Methods:

• Proposes a two-stage heterogeneous centralized training approach.
• Conducts two sequential training processes per time step: role-specific reward maximization and collective reward maximization.
• Applies the method to a 5 versus 5 AI robot soccer simulation using Webots.

Main Results:

• The proposed method effectively trains AI robot soccer teams.
• Achieves higher individual role rewards and team rewards compared to three other approaches.
• Demonstrates a 5% to 30% improvement in score-concede rate against evaluation teams.

Conclusions:

• The two-stage training method significantly improves cooperative learning in HMARL.
• Simultaneous optimization of role and team rewards leads to superior agent performance.
• The approach is validated as effective in complex simulated environments like robot soccer.