Collection and Analysis of Adherence Information for Software as a Medical Device Clinical Trials: Systematic Review

Area of Science:

• Digital Health
• Regulatory Science
• Clinical Trials

Background:

• Digital health apps require robust regulatory oversight for safety and effectiveness.
• Nonadherence and varied user engagement can skew trial results for digital health interventions.
• Current standards for measuring adherence and mitigating bias in efficacy analyses are lacking.

Purpose of the Study:

• To assess the reporting of adherence and engagement metrics in clinical trials of software as a medical device (SaMD) apps.
• To evaluate the appropriateness of efficacy analyses used to address nonadherence in SaMD trials.

Main Methods:

• Systematic review of Food and Drug Administration-registered, patient-facing SaMD therapeutics and their clinical trials.
• Analysis of 24 articles using a framework based on Cochrane risk-of-bias criteria.
• Summarization and analysis of adherence, engagement, and efficacy analysis data.

Main Results:

• Most articles (96%) collected engagement data, but only 45% reported comprehensive adherence for prescribed use.
• Significant variation existed in reported metrics for initiation, implementation, and persistence.
• Effectiveness estimates in most trials were potentially biased by nonadherence, with few (10%) using appropriate efficacy analysis methods.

Conclusions:

• Future SaMD trials need consistent and reliable adherence metrics.
• Preregistration of analyses and less biased efficacy analysis methods are recommended.
• Full reporting of statistical methods and assumptions is essential for SaMD trial transparency.