To warrant clinical adoption AI models require a multi-faceted implementation evaluation

Area of Science:

• Health Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support Systems

Background:

• Artificial intelligence (AI) technology is advancing rapidly.
• Translating AI advancements into clinical value and bedside adoption remains a challenge.
• Limited adoption of AI-based clinical decision support systems (CDSS) has been observed.

Purpose of the Study:

• To review the current use of implementation outcomes in randomized controlled trials (RCTs) for AI-based CDSS.
• To identify factors hindering the clinical adoption of AI.
• To propose a framework for improving the translation of AI into clinical value.

Main Methods:

• Literature review of randomized controlled trials (RCTs) evaluating AI-based clinical decision support.
• Analysis of implementation outcomes reported in the selected RCTs.
• Synthesis of findings to identify gaps in current research and practice.

Main Results:

• The review found limited adoption of AI-based clinical decision support systems in clinical practice.
• Implementation outcomes are not consistently reported or analyzed in RCTs evaluating AI CDSS.
• A gap exists between traditional quantitative metrics and implementation outcomes for AI systems.

Conclusions:

• To advance trust and clinical adoption of AI, a better understanding of implementation outcomes is needed.
• Bridging the gap between quantitative metrics and implementation outcomes is essential for AI success.
• Future research should focus on integrating implementation science principles into AI CDSS evaluations to improve clinical translation and value.