From Development to Implementation: A Systematic Review on the Current Maturity Status of Artificial Intelligence...

Introduction: Artificial intelligence (AI) is increasingly being researched and developed in the medical field and holds the potential to transform healthcare after successful implementation. For patients with colorectal cancer liver metastases (CRLM), many AI models have been developed, but knowledge about translation of these models in the clinical workflow is lacking. Therefore, this systematic review aimed to provide a contemporary overview of the current maturity status of AI models for patients with CRLM.

A systematic search of the literature until November 2, 2023, was conducted in PubMed, Embase.com, and Clarivate Analytics/Web of Science Core Collection to identify eligible studies. Studies using AI and/or radiomics for patients with CRLM were considered eligible. Data on the study aim, study design, size of dataset, country, type of AI application, level of validation and clinical implementation status (NASA technology readiness levels) were collected. Risk of bias and applicability of the individual studies were evaluated using the Prediction Model Risk of Bias Assessment Tool (PROBAST).

A total of 117 studies were included. Ninety-seven studies (83%) have been published in the last 5 years. The most common study design was retrospective (96%). Thirty-five studies (30%) utilized a dataset of fewer than 50 patients with CRLM. Internal validation was performed in 63% of the studies and external validation in 17%. The remaining studies did not report validation. Half of the studies were classified as high risk of bias. None of the included studies performed real-time testing, workflow integration, clinical testing, or clinical integration.

Although a rapid increase in research describing the development of AI models for patients with CRLM has been observed in recent years, not a single AI model has been translated into clinical practice.