NVIF: Neighboring Variational Information Flow for Cooperative Large-Scale Multiagent Reinforcement Learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Communication-based multiagent reinforcement learning (MARL) shows promise for agent cooperation.
• Existing MARL methods struggle with information redundancy and unstable training in large systems.

Purpose of the Study:

• To introduce Neighboring Variational Information Flow (NVIF) for enhanced communication in MARL.
• To address limitations of existing methods in large-scale and complex multiagent systems.

Main Methods:

• NVIF enhances communication by providing agents with a Maximum Information Set (MIS).
• A two-stage training mechanism stabilizes the process: offline pretraining of NVIF, followed by online policy training.
• NVIF compresses information into a compact latent state using neighboring communication.

Main Results:

• NVIF-Proximal Policy Optimization (PPO) demonstrates potential for promoting cooperation with agent-specific rewards.
• Experimental results show NVIF's superiority in both heterogeneous and homogeneous multiagent settings.
• The method shows potential for multitask learning applications.

Conclusions:

• NVIF offers a stable and effective approach to communication-based MARL.
• The proposed method improves cooperation and performance in large-scale multiagent systems.
• NVIF is a promising technique for multitask learning in MARL.