Predicting Early Dropout in a Digital Tobacco Cessation Intervention: Replication and Extension Study.

Area of Science:

• Digital health interventions
• Behavioral science
• Public health

Background:

• Early dropout from digital interventions hinders user retention and health outcomes.
• A prior metric using first-week login data predicted dropout in tobacco cessation interventions.
• Generalizing this metric to diverse digital interventions is needed to improve retention strategies.

Purpose of the Study:

• To replicate Bricker et al.'s early dropout prediction model using a large-scale, multimodal digital intervention.
• To determine if first-week engagement patterns can identify users most likely to drop out early.
• To inform the development of "rescue" interventions aimed at improving user retention.

Main Methods:

• Utilized data from 70,265 web users of the EX digital tobacco cessation intervention.
• Defined first-week engagement as page views or SMS responses within 7 days of registration.
• Defined early dropout as no engagement for the subsequent year; used regression models to predict dropout based on daily engagement patterns.

Main Results:

• The multivariate model achieved an AUC of 0.72 (95% CI 0.71-0.73), validating the prediction of early dropout.
• Univariate models showed increasing predictive ability (AUC) up to day 4 of the first week.
• Model sensitivity decreased while specificity increased with later-day engagement metrics.

Conclusions:

• First-week engagement effectively predicts early dropout across different digital intervention modalities and engagement metrics.
• This predictive validity supports using early engagement as a robust construct to address low retention.
• Future research should focus on applying this model to develop interventions that enhance user retention and health outcomes.