Comprehensive analysis of frameworks for evaluating artificial intelligence solutions in Healthcare: A descriptive...

Despite the lack of pricing and reimbursement guidelines for AI driven health technologies at majority of EU jurisdictions, there are some new examples of value assessment framework for digital solutions. To facilitate the process of homogenous guidelines formulation for those EU countries without any guidelines there is a need to review the current state of art regarding the assessment criteria proposed by various research groups. This will not only facilitate an understanding of the usability of available frameworks but also aid decision-makers in composing future guidelines for value of AI-based medical technologies assessment. Given the pervasive impact of the digital revolution and the resulting development of AI-based technologies in medicine, as well as emerging frameworks for their evaluation, there is a critical need for a descriptive analysis of existing frameworks, which is the main goal of this article.

A literature review was performed in accordance with recommendations in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Based on 2649 hits, 11 studies (13 articles) were selected for descriptive analysis. The evaluation criteria for AI solutions along with definitions extracted from the included articles were aggregated using a language model - ChatGPT (ver. 3.5) - to maintain objectivity in the process.

The result of our analysis is an aggregated framework comprising five domains (Clinical Assessment, Economics, Ethics, Safety, and Usability), within which 35 distinct criteria for evaluating AI products, along with their definitions, have been established.

According to the authors' knowledge, this framework is the most comprehensive, encompassing such a wide range of aspects. With this comprehensive approach, decision-makers can directly compare various technologies addressing the same problem, making informed decisions about which solution best meets their needs and requirements. In addition to its comprehensiveness, an important feature of our aggregated framework is that it can draw an insight to payers, healthcare providers, and technical teams.