Benchmarking for Bayesian Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Bayesian Reinforcement Learning (BRL) agents leverage prior knowledge to maximize rewards.
• Existing benchmarks for BRL algorithms are often limited in scope and applicability.
• A standardized and comprehensive comparison methodology is needed for BRL algorithm development.

Purpose of the Study:

• To address the limitations of current BRL algorithm benchmarks.
• To introduce a novel methodology for comparing BRL algorithms.
• To provide an open-source library for facilitating reproducible BRL research.

Main Methods:

• Defined a comparison criterion evaluating algorithm performance on large sets of Markov Decision Processes (MDPs) sampled from probability distributions.
• Incorporated analysis of computation time requirements to enable comparison of non-anytime algorithms.
• Developed an open-source library including test problems, prior distributions, and state-of-the-art BRL algorithms.

Main Results:

• The developed methodology provides a robust framework for evaluating BRL algorithms.
• The open-source library facilitates standardized benchmarking and reproducible research.
• Comparative analysis of seven state-of-the-art algorithms was performed and results were discussed.

Conclusions:

• The new methodology and library significantly advance the field of Bayesian Reinforcement Learning.
• This work enables more reliable and comprehensive evaluation of BRL algorithms.
• The open-source release promotes wider adoption and further development in BRL research.