Successes and failures: what are we measuring?

Area of Science:

• Health Services Research
• Digital Health Evaluation
• Telemedicine Implementation Science

Background:

• Perceptions of telemedicine and telehealth success are multifaceted, varying by context, time, and observer perspective.
• Existing literature reviews highlight common methodological weaknesses in telehealth evaluations.
• Telehealth initiatives are frequently funded as short-term projects, complicating long-term value assessment.

Purpose of the Study:

• To address the methodological shortcomings in current telehealth evaluation literature.
• To reconcile the need for long-term value assessment with the reality of short-term project funding and evaluation criteria.
• To propose a framework for analyzing similar telehealth studies to identify commonalities, differences, and success factors.

Main Methods:

• Review of existing evaluation literature on telemedicine and telehealth.
• Analysis of common methodological issues identified in previous telehealth studies.
• Conceptual development of a framework for comparative analysis of telehealth applications.

Main Results:

• Current evaluations often use short-term project criteria, which are insufficient for determining the long-term value of mature telehealth applications.
• A significant gap exists between the expected long-term value assessment and the interim findings from pilot projects.
• The development of analytical frameworks is crucial for synthesizing findings across similar telehealth studies.

Conclusions:

• A novel framework is proposed to enable systematic examination of similar telehealth studies (e.g., diabetic home care).
• This approach aims to extract commonalities and differences to better understand where telehealth is effective.
• The framework will help define the variables that constitute 'success' in mature telehealth applications, moving beyond pilot feasibility.