Sample-efficient multi-agent reinforcement learning with masked reconstruction.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Multi-Agent Systems

Background:

• Deep reinforcement learning (DRL) combines reinforcement learning (RL) and deep learning for complex decision-making.
• DRL faces challenges with sample efficiency, requiring extensive data and training time, especially in multi-agent reinforcement learning (MARL).

Purpose of the Study:

• To enhance sample efficiency in multi-agent reinforcement learning by introducing a masked reconstruction task.
• To address the fundamental limitations of DRL in multi-agent systems.

Main Methods:

• Proposed a novel approach combining a masked reconstruction task with QMIX, termed M-QMIX.
• Utilized the StarCraft II micromanagement benchmark with 11 diverse scenarios (easy, hard, very hard).
• Focused on demonstrating improved sample efficiency using a limited number of time steps per scenario.

Main Results:

• M-QMIX demonstrated superior performance compared to QMIX in eight out of 11 scenarios.
• The proposed method showed significant improvements in sample efficiency under time constraints.
• Experimental validation on StarCraft II confirmed the effectiveness of the approach.

Conclusions:

• The M-QMIX method effectively addresses the sample efficiency limitations of DRL in multi-agent systems.
• Integrating a masked reconstruction task is a viable strategy for improving MARL performance.
• The findings suggest a promising direction for developing more efficient MARL algorithms.