MuDE: Multi-agent decomposed reward-based exploration.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Multi-Agent Systems

Background:

• Cooperative multi-agent reinforcement learning (MARL) uses shared rewards and value function decomposition.
• Estimating individual agent contributions from rewards is crucial but challenging.
• Conflicting reinforcement and punishment signals complicate learning cooperative behaviors.

Purpose of the Study:

• To propose a novel exploration scheme, multi-agent decomposed reward-based exploration (MuDE).
• To address the challenge of reward decomposition and conflicting signals in MARL.
• To improve exploration of action spaces and cooperative behavior learning.

Main Methods:

• Developed MuDE, a modified reward decomposition scheme for preferential exploration of positive sub-rewards.
• Integrated MuDE into a MARL framework.
• Evaluated MuDE on StarCraft II micromanagement and predator-prey tasks with reinforcement and punishment.

Main Results:

• MuDE accurately estimates sub-rewards.
• MuDE effectively explores previously unreachable action spaces.
• MuDE demonstrates superior performance in convergence speed and win rates compared to state-of-the-art approaches.

Conclusions:

• MuDE offers an effective solution for reward decomposition and exploration in MARL.
• The proposed method enhances cooperative behavior learning in complex environments.
• MuDE represents a significant advancement in multi-agent reinforcement learning research.