Development of an evidence-based evaluation framework for digital health software products

Area of Science:

• Health Informatics
• Software Engineering
• Public Health Policy

Background:

• Existing digital health software product (DHSP) evaluation frameworks are fragmented.
• Current standards may not capture all stakeholder priorities for DHSP assessment.
• A unified, evidence-based approach is needed to evaluate DHSP quality and trustworthiness.

Purpose of the Study:

• To develop an evidence-based, open-access evaluation framework (EF) for digital health software products (DHSPs).
• To identify key components and domains for assessing DHSP quality and trustworthiness.
• To harmonize best practices and facilitate the adoption of high-quality DHSPs.

Main Methods:

• Mixed-methods study involving a needs assessment with 173 subject matter experts (SMEs) through interviews, focus groups, and surveys.
• Narrative review of 1053 papers and landscape analysis of 160 relevant frameworks, guidances, and standards.
• Validation of the developed EF through 58 additional SME interviews.

Main Results:

• The developed EF includes high-level themes (domains), evaluation criteria, and benchmarks for DHSP assessment.
• Top domains for assessing overall DHSP quality include evidence, usability, and privacy and security.
• Equity emerged as a cross-cutting theme across all evaluated domains.

Conclusions:

• The novel EF consolidates best practices across multiple trust and value domains for DHSPs.
• This framework supports the effective adoption of high-quality, trustworthy digital health software.
• The EF provides a harmonized, evidence-based approach to DHSP evaluation.