On the Difficulty of Predicting Engagement with Digital Health for Substance Use

Area of Science:

• Digital mental health
• Behavioral science
• Machine learning in healthcare

Background:

• Digital interventions show promise for substance use disorder treatment.
• High user dropout rates limit the effectiveness of digital mental health tools.
• Early prediction of engagement is needed to identify at-risk individuals and offer timely support.

Purpose of the Study:

• To investigate the predictability of user engagement with digital interventions for substance use disorder.
• To determine if baseline data from standardized psychometric measures can predict engagement with digital cognitive behavioral therapy (dCBT).

Main Methods:

• Utilized machine learning models to predict real-world engagement metrics.
• Employed baseline data from routinely collected, standardized psychometric measures as predictors.
• Evaluated prediction accuracy using Area Under the ROC Curve (AUC) and correlation analyses.

Main Results:

• Baseline data alone were insufficient to accurately predict individual engagement patterns.
• Machine learning models did not achieve high predictive accuracy for engagement metrics based on initial psychometric data.
• Observed and predicted values showed limited correlation, indicating poor model performance.

Conclusions:

• Standardized baseline psychometric data are inadequate for predicting engagement with digital substance use disorder interventions.
• Further research is needed to identify more effective predictors of user engagement.
• Developing accurate engagement prediction models is essential for optimizing digital intervention effectiveness.