Rethinking the dose-response relationship between usage and outcome in an online intervention for depression:...

There is now substantial evidence that Web-based interventions can be effective at changing behavior and successfully treating psychological disorders. However, interest in the impact of usage on intervention outcomes has only been developed recently. To date, persistence with or completion of the intervention has been the most commonly reported metric of use, but this does not adequately describe user behavior online. Analysis of alternative measures of usage and their relationship to outcome may help to understand how much of the intervention users may need to obtain a clinically significant benefit from the program.

The objective of this study was to determine which usage metrics, if any, are associated with outcome in an online depression treatment trial.

Cardiovascular Risk E-couch Depression Outcome (CREDO) is a randomized controlled trial evaluating an unguided Web-based program (E-couch) based on cognitive behavioral therapy and interpersonal therapy for people with depression and cardiovascular disease. In all, 280 participants in the active arm of the trial commenced the program, delivered in 12 modules containing pages of text and activities. Usage data (eg, number of log-ins, modules completed, time spent online, and activities completed) were captured automatically by the program interface. We estimated the association of these and composite metrics with the outcome of a clinically significant improvement in depression score on the Patient Health Questionnaire (PHQ-9) of ≥ 5 points.

In all, 214/280 (76.4%) participants provided outcome data at the end of the 12-week period and were included in the analysis. Of these, 94 (43.9%) participants obtained clinically significant improvement. Participants logged into the program an average of 18.7 times (SD 8.3) with most (62.1%, 133/214) completing all 12 modules. Average time spent online per log-in was 17.3 minutes (SD 10.5). Participants completed an average of 9 of 18 activities available within the program. In a multivariate regression model, only the number of activities completed per log-in was associated with a clinically significant outcome (OR 2.82, 95% CI 1.05-7.59). The final model predicted 7.4% of variance in outcome. Curve estimates indicated that significant logarithmic (P=.009) and linear (P=.002) relationships existed between activities completed per log-in and clinically significant change.

Only one objective measure of usage was independently associated with better outcome of a Web-based intervention of known effectiveness. The 4 usage metrics retained in the final step of the regression accounted for little outcome variance. Medium level users appeared to have little additional benefit compared to low users indicating that assumptions of a linear relationship between use and outcome may be too simplistic and further models and variables need to be explored to adequately understand the relationship.