Generalized Contextual Bandits With Latent Features: Algorithms and Applications.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Contextual bandits are crucial for balancing exploration-exploitation in applications like recommender systems.
• Real-world scenarios involve latent contexts and human biases in feedback, necessitating advanced frameworks.

Purpose of the Study:

• To develop a generalized contextual bandit framework addressing latent contexts and human biases.
• To propose and analyze algorithms for efficient and accurate sequential decision-making.

Main Methods:

• Formulated a two-layer probabilistic interpretable model for human feedback with latent features.
• Designed the GCL-PS algorithm using posterior sampling and proved its sublinear regret.
• Developed the GCL-PSMC algorithm with MCMC for improved computational efficiency and analyzed its Bayesian regret.

Main Results:

• Proved sublinear regret upper bounds for GCL-PS and Bayesian regret for GCL-PSMC.
• Demonstrated the optimality insights of GCL-PS through lower bound analysis.
• Showcased superior performance of the proposed framework in hotel and news recommendations via experiments.

Conclusions:

• The generalized contextual bandit framework effectively handles latent contexts and human biases.
• GCL-PSMC offers a favorable tradeoff between computational efficiency and decision accuracy.
• The framework shows practical utility and superior performance in real-world recommendation tasks.