Representation-driven sampling and adaptive policy resetting for improving multi-Agent reinforcement learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multi-agent reinforcement learning (MARL) is crucial for complex decision-making but faces challenges like policy convergence and sampling inefficiency.
• Large state, observation, and action spaces exacerbate these issues, leading to suboptimal strategies and extensive training requirements.

Purpose of the Study:

• To propose a novel MARL approach, eXJTU-MARL, enhancing exploration and trajectory learning efficiency.
• To address insufficient exploration and sampling inefficiency in mainstream MARL algorithms.

Main Methods:

• Introduction of adaptive policy resetting to prevent premature convergence.
• Development of state representation-based balanced experience sampling for improved data efficiency.
• Implementation within the eXJTU-MARL framework for multi-agent decision-making tasks.

Main Results:

• eXJTU-MARL significantly enhances sample efficiency and exploration capabilities.
• The approach prevents premature convergence to suboptimal policies.
• Experiments in StarCraft Multi-Agent Challenge show superior performance over existing MARL baselines.

Conclusions:

• eXJTU-MARL effectively improves exploration and learning efficiency in complex MARL environments.
• Adaptive policy resetting and balanced experience sampling are key contributions.
• The proposed method offers a robust solution for advanced multi-agent decision-making.