Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling.

Area of Science:

• Digital Health Interventions
• Machine Learning in Healthcare
• Behavioral Science

Background:

• Growing interest in using reinforcement learning (RL) for personalized digital health treatments to promote healthier behaviors.
• Sequential decision-making in digital health relies on user context for treatment timing and type.
• Online RL offers a data-driven approach to personalize treatments based on user responses.

Purpose of the Study:

• To assess the data evidence confirming that RL algorithms genuinely personalize treatments for users.
• To investigate if observed RL personalization is an artifact of the algorithm's stochasticity.
• To introduce a resampling-based methodology for evaluating RL personalization.

Main Methods:

• Developed a working definition of personalization for RL algorithms.
• Introduced a resampling-based methodology to distinguish genuine personalization from algorithmic stochasticity.
• Applied the methodology to data from the HeartSteps physical activity clinical trial.

Main Results:

• Demonstrated a method to rigorously evaluate the personalization capabilities of online RL algorithms.
• The case study illustrated how the approach enhances the assessment of algorithm personalization.
• Personalization was evaluated both across all users and within individual user data.

Conclusions:

• The proposed methodology provides a framework for data-driven truth-in-advertising of RL personalization in digital health.
• Validating RL personalization is crucial for the ethical and effective deployment of optimized digital health interventions.
• This approach helps ensure that RL algorithms provide meaningful, individualized support for behavior change.