Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Distributed Multi-Agent Reinforcement Learning (MARL) faces challenges with conventional algorithms due to fixed reward models.
• Deep Neural Network (DNN)-based MARL solutions often suffer from overfitting, parameter sensitivity, and sample inefficiency.

Purpose of the Study:

• To introduce an adaptive Kalman Filter (KF)-based framework as an efficient alternative for MARL problems.
• To address limitations of existing MARL algorithms, particularly in high-dimensional, continuous action-space environments.

Main Methods:

• Proposed the Multi-Agent Adaptive Kalman Temporal Difference (MAK-TD) framework and its Successor Representation variant (MAK-SR).
• Utilized Kalman Temporal Difference (KTD) to manage parameter uncertainty and leverage KF's uncertainty modeling and online second-order learning.
• Evaluated frameworks on OpenAI Gym MARL benchmarks across cooperative, competitive, and mixed scenarios.

Main Results:

• The proposed MAK-TD/SR frameworks demonstrated superior performance compared to state-of-the-art MARL counterparts.
• Effectiveness shown across various multi-agent scenarios with differing numbers of agents.

Conclusions:

• The adaptive KF-based MAK-TD/SR frameworks offer an efficient and robust solution for complex MARL problems.
• These novel frameworks overcome key limitations of traditional and DNN-based MARL approaches.