Usage Metrics of Web-Based Interventions Evaluated in Randomized Controlled Trials: Systematic Review

Area of Science:

• Digital Health Interventions
• Clinical Trial Methodology
• Health Informatics

Background:

• Web-based interventions are increasingly evaluated in randomized controlled trials (RCTs).
• Current understanding of how participant usage of these interventions is measured, reported, and analyzed in RCTs is limited.

Purpose of the Study:

• To systematically review the evaluation of web-based interventions within RCTs.
• To assess study characteristics and methods for recording and adjusting for intervention usage.

Main Methods:

• Systematic literature review of RCTs involving web-based interventions.
• Random sample of 100 published trials selected for detailed data extraction.
• Analysis of trial characteristics, web usage data collection, and reporting in efficacy analyses.

Main Results:

• A growing trend in web-based intervention RCTs was observed (812 identified up to 2017).
• Of 100 reviewed studies, 90 collected web usage data, but 54% did not report the method.
• Only 21% adjusted for intervention use in outcomes, with just two using appropriate statistical methods.

Conclusions:

• Reporting of web-based intervention usage methods and analysis details in RCTs is often insufficient.
• Appropriate statistical methods to account for intervention use are rarely employed or well-reported.
• Reliability checks for web usage data collection methods are seldom performed.