Meta Learning Task Representation in Multiagent Reinforcement Learning: From Global Inference to Local Inference.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multiagent meta reinforcement learning (MAMRL) allows multiagent systems (MASs) to adapt across diverse tasks.
• Partial observability in MASs significantly challenges efficient task inference due to limited local agent experiences.
• Existing methods struggle to effectively bridge the gap between global system knowledge and local agent observations.

Purpose of the Study:

• Introduce MG2L, a novel algorithm for MAMRL under partial observability.
• Develop a global-to-local (G2L) training scheme leveraging mutual information optimization (MIO).
• Enhance task inference capabilities for improved agent adaptability and performance.

Main Methods:

• Extend the centralized training and decentralized execution (CTDE) framework for MAMRL.
• Propose a multilevel task encoder for joint global and local task inference.
• Utilize mutual information (MI) maximization for global representation and conditional MI reduction for local representation learning.
• Integrate a permutation-invariant attention (PIA) module to mitigate policy variation sensitivity.

Main Results:

• MG2L effectively harmonizes centralized training with decentralized execution in MAMRL.
• The G2L scheme successfully improves task inference accuracy and agent adaptability.
• Experiments demonstrate significant performance gains and robustness compared to baseline methods.
• Ablation studies and visualizations validate the contribution of individual components.

Conclusions:

• MG2L offers a versatile and effective solution for MAMRL challenges, particularly under partial observability.
• The proposed G2L training scheme and task encoder advance the state-of-the-art in adaptive multiagent systems.
• The publicly available implementation facilitates further research and application of the MG2L algorithm.