e-Mental Health Program Usage Patterns in Randomized Controlled Trials and in the General Public to Inform External...

Area of Science:

• Digital mental health
• Clinical psychology
• Health services research

Background:

• Randomized controlled trials (RCTs) are crucial for treatment efficacy but face external validity challenges.
• Self-selection bias in RCTs can lead to overestimations of treatment adherence and engagement.
• These concerns are particularly relevant for unguided e-mental health (eMH) tools intended for broad public access.

Purpose of the Study:

• To compare engagement patterns between participants in an eMH RCT and community users of the same self-guided program.
• To identify distinct user engagement profiles in both RCT and community samples.
• To assess whether usage styles or frequency differ between evaluation and naturalistic contexts.

Main Methods:

• Clustering techniques were used to analyze user engagement data from RCT participants and community users of the myCompass eMH program.
• Usage metrics (logins, trackings, activities) were collected over the initial 49 days post-registration.
• Demographic data and engagement patterns were compared between the two groups.

Main Results:

• Community users were older on average than RCT participants.
• RCT participants demonstrated significantly higher engagement frequency across all metrics compared to community users within the first 49 days.
• Cluster analyses revealed comparable usage patterns but differing magnitudes of engagement, with RCT groups showing higher frequency.

Conclusions:

• External validity concerns in eMH RCTs may relate more to the magnitude of engagement than to usage styles.
• Investigating non-starters in RCTs could reveal reasons for disengagement with eMH tools.
• Overestimating eMH tool engagement frequency has implications for theoretical understanding and public dissemination strategies.