Measuring Engagement in Provider-Guided Digital Health Interventions With a Conceptual and Analytical Framework Using...

Area of Science:

• Digital Health Interventions
• Behavioral Science
• Oncology Nursing

Background:

• Limited guidance exists for analyzing participant engagement in provider-guided digital health interventions (DHIs).
• Commonly assessed system usage has limitations in measuring socio-affective and cognitive engagement aspects.
• The Nurse WRITE DHI provides an opportunity to develop a multidimensional engagement assessment framework.

Purpose of the Study:

• To develop a conceptual and analytic framework for measuring socio-affective, cognitive, and behavioral engagement in provider-guided DHIs.
• To illustrate the framework's utility in describing and categorizing engagement using the Nurse WRITE intervention as a case study.

Main Methods:

• Adapted a prior engagement framework across three dimensions, finalizing six distinct measures.
• Developed socio-affective (post count, word count) and cognitive (question count, completion rate) engagement measures from participant posts.
• Utilized website data for behavioral engagement measures (care plans, reviews) and k-Means clustering to categorize participants by engagement levels.

Main Results:

• Participants were clustered into high (n=13), moderate (n=17), and low (n=38) engagement groups based on six measures.
• High engagers exhibited significantly higher socio-affective and cognitive engagement and plan completion rates compared to moderate and low engagers.
• Socio-affective engagement averaged 34.7 instances with 14,851 words; cognitive engagement averaged 19.4 instances with 78.3% inquiry completion.

Conclusions:

• Developed and reported an engagement framework to guide behavioral intervention scientists in analyzing participant engagement with provider-guided DHIs.
• Significant variations in engagement across socio-affective, cognitive, and behavioral dimensions underscore the importance of multidimensional assessment.
• Future research should validate the framework across diverse DHIs and explore patient/provider factors influencing engagement and intervention efficacy.