Multiagent Reinforcement Learning With Sparse Interactions by Negotiation and Knowledge Transfer.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multiagent reinforcement learning (MARL) faces challenges with computational complexity in large state-action spaces, hindering scalability in dynamic environments.
• Existing MARL algorithms struggle to efficiently coordinate multiple agents, especially in scenarios with sparse interactions.

Purpose of the Study:

• To present a novel algorithm, negotiation-based MARL with sparse interactions (NegoSI), addressing scalability and coordination issues in MARL.
• To introduce an equilibrium-based framework enabling agents to select optimal joint actions through negotiation.

Main Methods:

• NegoSI integrates an equilibrium-based framework for sparse interactions, negotiation for equilibrium set selection, minimum variance for joint action selection, and knowledge transfer of local Q-values.
• The algorithm employs unshared value functions for privacy, equilibrium solutions for coordination, and sparse interactions for reduced computational complexity.

Main Results:

• Experiments on grid world games showed NegoSI exhibits fast convergence and high scalability.
• Application to an intelligent warehouse problem demonstrated NegoSI's effectiveness compared to state-of-the-art MARL algorithms.
• Performance was evaluated based on steps per episode, rewards per episode, and average runtime.

Conclusions:

• NegoSI effectively addresses the scalability and coordination challenges in multiagent reinforcement learning.
• The algorithm's novel approach using negotiation and equilibrium concepts offers a promising solution for complex dynamic environments.
• NegoSI provides a privacy-preserving and computationally efficient alternative for advanced MARL applications.