A Robust Mixed-Effects Bandit Algorithm for Assessing Mobile Health Interventions.

Area of Science:

• Computer Science
• Machine Learning
• Health Informatics

Background:

• Mobile health (mHealth) utilizes personalized interventions optimized by bandit and reinforcement learning.
• Challenges in mHealth include participant heterogeneity, nonstationarity, and nonlinear rewards, which limit algorithm efficacy.

Purpose of the Study:

• To propose a robust contextual bandit algorithm, DML-TS-NNR, designed to overcome key challenges in mHealth intervention optimization.
• To enhance the performance of personalized, contextually-tailored interventions in mobile health applications.

Main Methods:

• The DML-TS-NNR algorithm models differential rewards using user- and time-specific parameters.
• It incorporates network cohesion penalties and debiased machine learning for flexible baseline reward estimation.
• A high-probability regret bound is established, dependent on the differential reward model's dimension.

Main Results:

• The algorithm achieves robust regret bounds, even with complex baseline reward structures.
• Superior performance of DML-TS-NNR was demonstrated through simulations.
• The algorithm's effectiveness was further validated in two off-policy evaluation studies.

Conclusions:

• DML-TS-NNR offers a robust solution for optimizing mHealth interventions by effectively handling participant heterogeneity and complex reward dynamics.
• The proposed method provides a flexible and powerful framework for advancing personalized mobile health strategies.
• The algorithm's performance highlights its potential for real-world application in adaptive mHealth systems.