Approaches to Evaluating Digital Health Technologies: Scoping Review

Area of Science:

• Digital Health Technologies
• Health Informatics
• Technology Evaluation

Background:

• Scientific evaluation of novel digital health technologies (DHTs) is crucial for successful development and implementation.
• The previously developed eHealth evaluation cycle comprises five phases: conceptual, development, feasibility, effectiveness, and implementation.
• Understanding the practical application of this cycle is essential for advancing DHT research.

Purpose of the Study:

• To analyze the practical application of the eHealth evaluation cycle's study phases through a structured literature review.
• To identify and categorize the evaluation approaches employed within each phase of the DHT evaluation cycle.
• To provide insights into current practices for evaluating digital health interventions.

Main Methods:

• A systematic literature search was conducted in PubMed using "telemedicine" and various evaluation approach terms.
• Original, peer-reviewed studies published in 2019 (pre-COVID-19) focusing on patient-centered evaluations were included.
• Data extraction and analysis encompassed journal, country, specialty, user, functionality, evaluation phase, and approach, utilizing RStudio for statistical analysis.

Main Results:

• Out of 1583 screened titles, 824 studies were included, with a primary focus on the effectiveness phase (36.9%) and minimal attention to implementation (8.5%).
• Randomized controlled trials (RCTs) were the most common evaluation method (18.9%), frequently used within the effectiveness phase.
• Survey and interview studies dominated the conceptual and planning phases, while the United States led in publications, with psychiatry/mental health and cardiology as leading specialties.

Conclusions:

• This study presents the first comprehensive overview of digital health technology evaluation practices across consecutive study phases.
• A significant imbalance exists in the evaluation of eHealth study phases, with effectiveness receiving the most attention.
• Achieving successful DHT development and implementation necessitates a more equitable focus on all evaluation phases and the selection of appropriate, iterative evaluation methods.