Efficient Exploration for Multi-Agent Diversity With Agent Identity.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multi-Agent Reinforcement Learning (MARL) enables decentralized policies via centralized training.
• Parameter sharing in MARL can cause agent behavioral similarity, limiting exploration.
• Existing diversity methods may overfit, hindering novel trajectory discovery.

Purpose of the Study:

• To introduce Active Exploration with Agent-Identity (AEAI) for improved multi-agent diversity and exploration.
• To address overfitting issues in current multi-agent diversity techniques.
• To enhance the learning of agent identity-conditioned policies.

Main Methods:

• AEAI maximizes trajectory entropy across agents for exploration.
• A novel lower bound for mutual information, using successor features, aligns trajectories and agent identities.
• The method is integrated with existing MARL algorithms.

Main Results:

• AEAI consistently outperforms state-of-the-art methods on challenging MARL benchmarks.
• The proposed method effectively fosters diversity among agents.
• Significant improvements in exploration efficiency were observed.

Conclusions:

• AEAI is a novel and effective exploration method for MARL.
• The approach successfully promotes agent diversity and enhances exploration.
• AEAI offers a promising direction for advancing MARL capabilities.