Predicting Long-Term Engagement in mHealth Apps: Comparative Study of Engagement Indices

Area of Science:

• Digital Health
• Health Informatics
• Mobile Health Interventions

Background:

• Digital health apps offer potential for increased accessibility and patient engagement beyond traditional healthcare.
• Current tools for quantitatively measuring long-term engagement in digital therapeutics are lacking.

Purpose of the Study:

• To evaluate an existing engagement index (EI) for long-term use in a commercial health management app.
• To compare the performance of the existing EI with a newly developed EI.

Main Methods:

• 240 cancer survivors from a randomized controlled trial using the Noom app were included.
• A newly developed EI was created, adapting measurements from the Web Matrix Visitor Index (click depth, recency, loyalty).
• The new EI was compared against the existing EI for predictive accuracy of long-term engagement.

Main Results:

• The newly developed EI model demonstrated superior predictive performance compared to the existing model (MSE 0.025 vs. 0.096, R² 0.610 vs. 0.053).
• The existing EI showed significant associations with survival: click depth (HR 0.49) and loyalty (HR 0.17).
• The best-performing new EI model, utilizing free version app menus, showed significant associations with loyalty (HR 0.32) and recency (HR 0.47).

Conclusions:

• The newly developed EI model is more effective for predicting long-term user engagement and compliance in mobile health apps.
• Log data is crucial for EI evaluation, and future research should address subjectivity and incorporate broader indices.