Comprehensive analysis of frameworks for evaluating artificial intelligence solutions in Healthcare: A descriptive...

Area of Science:

• Health Informatics
• Artificial Intelligence in Medicine
• Health Technology Assessment

Background:

• Limited pricing and reimbursement guidelines exist for AI-driven health technologies across EU jurisdictions.
• Emerging value assessment frameworks for digital health solutions highlight a need for standardized evaluation criteria.
• The rapid advancement of AI in medicine necessitates a review of current assessment frameworks.

Purpose of the Study:

• To conduct a descriptive analysis of existing frameworks for assessing AI-based medical technologies.
• To identify and aggregate assessment criteria proposed by various research groups.
• To aid EU countries in formulating homogenous guidelines for AI health technology value assessment.

Main Methods:

• A literature review was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.
• Eleven studies (13 articles) were selected from 2649 initial search results for descriptive analysis.
• Evaluation criteria and definitions were aggregated using a language model (ChatGPT ver. 3.5) for objectivity.

Main Results:

• An aggregated framework was developed, encompassing five key domains: Clinical Assessment, Economics, Ethics, Safety, and Usability.
• A total of 35 distinct criteria for evaluating AI products, along with their definitions, were established within these domains.
• This framework represents a comprehensive approach to evaluating AI-driven health technologies.

Conclusions:

• The developed framework is the most comprehensive to date, covering a wide range of assessment aspects for AI health technologies.
• It enables direct comparison of technologies addressing similar problems, supporting informed decision-making by stakeholders.
• The framework provides valuable insights for payers, healthcare providers, and technical teams involved in AI adoption.