Assessing User Engagement of an mHealth Intervention: Development and Implementation of the Growing Healthy App Engagement Index
Related Experiment Video
Updated: Jun 12, 2026
Measuring Engagement of Spectators of Social Digital Games
Published on: July 3, 2021
Sarah Taki1,2, Sharyn Lymer2,3, Catherine Georgina Russell2,4
1University of Technology Sydney, Sydney, Australia.
Insights
This study developed an engagement index (EI) to measure user interaction with the Growing healthy mobile health app. The EI effectively captured participant behavior, informing future mHealth intervention improvements for childhood obesity prevention.
Area of Science:
- Public Health
- Digital Health
- Health Informatics
Background:
- Childhood obesity is a significant public health issue, particularly affecting socioeconomically disadvantaged families.
- Mobile health (mHealth) interventions present a scalable solution for delivering targeted health information to at-risk populations.
- The Growing healthy program utilized an app to provide infant feeding guidance from birth to nine months.
Purpose of the Study:
- To develop and describe an engagement index (EI) for monitoring user interaction with the Growing healthy mobile application.
- To establish cut-points for categorizing user engagement levels within the mHealth intervention.
- To assess the utility of the EI in understanding participant behavior and informing intervention design.
Main Methods:
- A feasibility study involving 300 mother-infant dyads using the Growing healthy app with weekly push notifications.
- Data collection included surveys at three time points (infant age ≤3, 6, and 9 months) and app usage metrics.
- The engagement index (EI) incorporated five subindices: click depth, loyalty, interaction, recency, and feedback, calculated over 39 weeks.
Main Results:
- The overall mean engagement index (EI) score was 30.0%, with high engagement defined as >37.1% and poor engagement as <21.1%.
- Significant predictors of EI scores included parity, system type (app only vs. app and email), recruitment method, and infant age at recruitment.
- The EI demonstrated a comprehensive understanding of participant engagement throughout the 9-month intervention period.
Conclusions:
- The developed engagement index (EI) offers a robust method for assessing user interaction with mHealth applications.
- The EI provides valuable insights into the strengths and weaknesses of digital health interventions.
- This approach can inform the iterative development and optimization of future mHealth programs for childhood obesity prevention.
Background:
Childhood obesity is an ongoing problem in developed countries that needs targeted prevention in the youngest age groups. Children in socioeconomically disadvantaged families are most at risk. Mobile health (mHealth) interventions offer a potential route to target these families because of its relatively low cost and high reach. The Growing healthy program was developed to provide evidence-based information on infant feeding from birth to 9 months via app or website. Understanding user engagement with these media is vital to developing successful interventions. Engagement is a complex, multifactorial concept that needs to move beyond simple metrics.
Objective:
The aim of our study was to describe the development of an engagement index (EI) to monitor participant interaction with the Growing healthy app. The index included a number of subindices and cut-points to categorize engagement.
Methods:
The Growing program was a feasibility study in which 300 mother-infant dyads were provided with an app which included 3 push notifications that was sent each week. Growing healthy participants completed surveys at 3 time points: baseline (T1) (infant age ≤3 months), infant aged 6 months (T2), and infant aged 9 months (T3). In addition, app usage data were captured from the app. The EI was adapted from the Web Analytics Demystified visitor EI. Our EI included 5 subindices: (1) click depth, (2) loyalty, (3) interaction, (4) recency, and (5) feedback. The overall EI summarized the subindices from date of registration through to 39 weeks (9 months) from the infant's date of birth. Basic descriptive data analysis was performed on the metrics and components of the EI as well as the final EI score. Group comparisons used t tests, analysis of variance (ANOVA), Mann-Whitney, Kruskal-Wallis, and Spearman correlation tests as appropriate. Consideration of independent variables associated with the EI score were modeled using linear regression models.
Results:
The overall EI mean score was 30.0% (SD 11.5%) with a range of 1.8% - 57.6%. The cut-points used for high engagement were scores greater than 37.1% and for poor engagement were scores less than 21.1%. Significant explanatory variables of the EI score included: parity (P=.005), system type including "app only" users or "both" app and email users (P<.001), recruitment method (P=.02), and baby age at recruitment (P=.005).
Conclusions:
The EI provided a comprehensive understanding of participant behavior with the app over the 9-month period of the Growing healthy program. The use of the EI in this study demonstrates that rich and useful data can be collected and used to inform assessments of the strengths and weaknesses of the app and in turn inform future interventions.
