Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity.

Area of Science:

• Digital Health
• Behavioral Science
• Machine Learning

Background:

• Mobile health technologies are rapidly advancing, leading to increased interest in just-in-time adaptive interventions (JITAIs).
• JITAIs utilize mobile device notifications to deliver timely support for health behavior change.
• Effective JITAIs require dynamic treatment policies that adapt to individual user context.

Purpose of the Study:

• To describe a novel reinforcement learning (RL) algorithm for continuously improving JITAI treatment policies.
• To apply this RL algorithm to the HeartSteps V2 mobile physical activity application.
• To optimize the delivery of context-tailored activity suggestions for improved health outcomes.

Main Methods:

• Development of a reinforcement learning (RL) algorithm to learn optimal treatment policies.
• Sequential decision-making framework using user context to guide intervention delivery.
• Application of the RL algorithm to real-world data from the HeartSteps V1 and V2 physical activity interventions.

Main Results:

• The RL algorithm continuously learns and refines the JITAI treatment policy as new user data becomes available.
• The developed RL algorithm is implemented in HeartSteps V2 to make real-time decisions on activity suggestions.
• The system is designed to deliver context-tailored suggestions five times daily.

Conclusions:

• Reinforcement learning offers a powerful approach for optimizing just-in-time adaptive interventions (JITAIs) in mobile health.
• Continuous learning algorithms can enhance the effectiveness of digital health tools by adapting to user needs.
• This work demonstrates a practical application of RL for promoting physical activity through mobile health technology.