Metrics for Evaluating Telemedicine in Randomized Controlled Trials: Scoping Review

Telemedicine involves medical, diagnostic, and treatment-related services using telecommunication technology. Not only does telemedicine contribute to improved patient quality of life and satisfaction by reducing travel time and allowing patients to be seen in their usual environment, but it also has the potential to improve disease management by making it easier for patients to see a doctor. Recently, owing to IT developments, research on telemedicine has been increasing; however, its usefulness and limitations in randomized controlled trials remain unclear because of the multifaceted effects of telemedicine. Furthermore, the specific metrics that can be used as cross-disciplinary indicators when comparing telemedicine and face-to-face care also remain undefined.

This review aimed to provide an overview of the general and cross-disciplinarity metrics used to compare telemedicine with in-person care in randomized controlled trials. In addition, we identified previously unevaluated indicators and suggested those that should be prioritized in future clinical trials.

MEDLINE and Embase databases were searched for publications that met the inclusion criteria according to PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews). Original, English-language articles on randomized controlled trials comparing some forms of telemedicine with face-to-face care from January 2019 to March 2024 were included, and the basic information and general metrics used in these studies were summarized.

Of the 2275 articles initially identified, 79 were included in the final analysis. The commonly used metrics that can be used across medical specialties were divided into the following 3 categories: (1) patient-centeredness (67/79, 85%), including patient satisfaction, workload, and quality of life; (2) patient outcomes (57/79, 72%), including general clinical parameters such as death, admission, and adverse events; and (3) cost-effectiveness (40/79, 51%), including cost assessment and quality-adjusted life year. Notably, only 25 (32%) of 79 studies evaluated all the 3 categories. Other metrics, such as staff convenience, system usability, and environmental impact, were extracted as indicators in different directions from the three categories above, although few previous reports have evaluated them (staff convenience: 8/79, 10%; system usability: 3/79, 4%; and environmental impact: 2/79, 3%).

A significant variation was observed in the metrics used across previous studies. Notably, general indicators should be used to enhance the understandability of the results for people in other areas, even if disease-specific indicators are used. In addition, indicators should be established to include all three commonly used categories of measures to ensure a comprehensive evaluation: patient-centeredness, patient outcomes, and cost-effectiveness. Staff convenience, system usability, and environmental impact are important indicators that should be used in future trials. Moreover, standardization of the evaluation metrics is desired for future clinical trials and studies.

Open Science Forum Registries YH5S7; https://doi.org/10.17605/OSF.IO/YH5S7.