Action Centered Contextual Bandits.

Area of Science:

• Machine Learning
• Mobile Health Applications
• Reinforcement Learning

Background:

• Contextual bandits balance multi-armed bandits and reinforcement learning, showing success in web applications.
• Linear models for contextual bandits are interpretable and theoretically sound but assume time-invariant relationships.
• The time-invariant assumption is often unsuitable for dynamic mobile health applications.

Purpose of the Study:

• To extend linear models for contextual bandits to address the limitations of time-invariant assumptions in mobile health.
• To develop a model that accommodates complex, nonlinear baseline rewards while maintaining a simple treatment effect.
• To ensure strong theoretical performance guarantees comparable to traditional linear models.

Main Methods:

• Proposed a two-part extension to linear contextual bandit models: a complex baseline reward component and a simple treatment effect component.
• Developed algorithms based on this extended model, designed for mobile health scenarios.
• Validated the theoretical framework with experimental analysis on mobile health study data.

Main Results:

• The proposed model is plausible for mobile health applications, allowing for nonlinear baseline reward modeling.
• Algorithms derived from the extended model achieve strong performance guarantees.
• Experimental results on mobile health data support the theoretical findings.

Conclusions:

• The extended linear model offers a viable approach for contextual bandits in mobile health where time-varying dynamics are prevalent.
• This model provides a practical solution balancing model complexity and performance guarantees.
• The findings suggest improved decision-making strategies in mobile health interventions.