Optimal Query Selection Using Multi-Armed Bandits.

Area of Science:

• Machine Learning
• Control Systems
• Signal Processing

Background:

• Conventional query selection for latent variable estimation relies on low-noise observations or information-theoretic objectives.
• Current methods can be slow and inaccurate when the system's estimate deviates significantly from the true state.

Purpose of the Study:

• To develop a novel sequential adaptive action value function for improved query selection in latent variable estimation.
• To address limitations of current methods that rely on the current best estimate.

Main Methods:

• Utilized the multi-armed bandit (MAB) framework to create a sequential adaptive action value function for query selection.
• Employed analytical methods to demonstrate performance improvements and identify conditions for outperforming competitors.
• Conducted Monte Carlo simulations and human-in-the-loop experiments with a brain-computer interface (BCI) typing system.

Main Results:

• Demonstrated analytically that the proposed method improves query selection performance in dynamical systems.
• Identified specific conditions under which the new model outperforms existing approaches.
• Empirical assessments showed favorable performance compared to alternative methods in simulations and BCI experiments.

Conclusions:

• The novel sequential adaptive query selection method offers significant improvements in latent variable estimation.
• The multi-armed bandit approach provides a tractable and effective solution for adaptive query selection.
• The method shows promise for applications like brain-computer interface systems.