SATF: A Scalable Attentive Transfer Framework for Efficient Multiagent Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Training multiagent reinforcement learning (MARL) models becomes computationally expensive as the number of agents increases, leading to exponentially expanding observation spaces.
• Large-scale multiagent systems face significant challenges in maintaining learning efficiency and scalability due to these complex observation spaces.

Purpose of the Study:

• To propose a scalable attentive transfer framework (SATF) for efficient MARL in large-scale systems.
• To enhance MARL performance by enabling knowledge transfer from a smaller agent set to a larger one.
• To address the challenge of increasing observation space dimensionality with agent count.

Main Methods:

• Developed a novel dynamic observation representation network (DorNet) utilizing a self-attention mechanism to extract dominant observed information cost-effectively.
• Implemented the SATF to reduce and align state representation dimensionality, accommodating varying numbers of agents.
• Evaluated the framework on homogeneous and heterogeneous combat tasks using the MAgent and StarCraft II platforms.

Main Results:

• The SATF demonstrated superior performance compared to distributed MARL methods like independent Q-learning (IQL) and A2C on the MAgent platform with 8 to 64 agents.
• On StarCraft II, SATF outperformed QMIX, achieving up to 90% win rate with 32 agents and requiring fewer training steps.
• The framework achieved goals faster and more accurately by effectively transferring learned knowledge from 4 to 64 agents.

Conclusions:

• The scalable attentive transfer framework (SATF) significantly enhances MARL efficiency in large-scale combat missions.
• DorNet's self-attention mechanism effectively manages complex state representations, proving cost-effective.
• The findings highlight SATF's potential for improving MARL training efficiency in complex, large-scale agent environments.