Master-Slave Deep Architecture for Top-K Multiarmed Bandits With Nonlinear Bandit Feedback and Diversity Constraints

Area of Science:

• Machine Learning
• Artificial Intelligence
• Optimization

Background:

• Combinatorial multi-armed bandits (CMABs) are crucial for sequential decision-making.
• Existing CMABs often lack efficient handling of nonlinear feedback and diversity constraints.
• Addressing these limitations is key for improving recommendation systems.

Purpose of the Study:

• To propose a novel master-slave architecture for the top-K CMABs problem.
• To incorporate nonlinear bandit feedback and diversity constraints effectively.
• To enhance exploration-exploitation trade-offs in complex action spaces.

Main Methods:

• Developed a master-slave architecture with six specialized slave models.
• Implemented teacher learning-based optimization and policy cotraining for slave models.
• Utilized a NeuralUCB-based master model for sample selection and decision-making.

Main Results:

• The proposed approach significantly outperforms state-of-the-art algorithms.
• Demonstrated superior performance on both synthetic and real-world recommendation datasets.
• The architecture effectively balances rewards, constraints, and exploration efficiency.

Conclusions:

• The novel master-slave architecture is highly effective for top-K CMABs with diversity constraints.
• The combination of specialized slave models and master-slave interaction yields significant performance gains.
• This work provides a robust framework for complex bandit problems in recommendation systems.