Multi-Agent Thompson Sampling for Bandit Applications with Sparse Neighbourhood Structures.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Theory

Background:

• Multi-agent coordination is vital but complex due to combinatorial challenges.
• Real-world agents often exhibit loose couplings, affecting only neighbors.
• Leveraging these loose couplings is crucial for feasible coordination.

Purpose of the Study:

• To develop an efficient coordination algorithm for loosely-coupled agents in a multi-agent system.
• To address the multi-agent multi-armed bandit framework for cooperative decision-making.
• To propose a novel Bayesian exploration-exploitation algorithm.

Main Methods:

• Propose multi-agent Thompson sampling (MATS), a Bayesian exploration-exploitation algorithm.
• Analyze MATS using regret bounds for sparse coordination graphs.
• Empirically evaluate MATS on synthetic and Poisson distribution benchmarks.

Main Results:

• MATS achieves sublinear regret bounds in time.
• Demonstrates superior performance compared to the state-of-the-art MAUCE algorithm.
• Successfully applied to a realistic wind farm control task, optimizing power production.

Conclusions:

• MATS effectively leverages loose couplings for efficient multi-agent coordination.
• The algorithm shows significant performance improvements in practical applications.
• MATS offers a valuable tool for optimizing systems with sparse neighborhood structures.