Identifying and evaluating barriers for the implementation of machine learning in the intensive care unit

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Machine Learning for Healthcare

Background:

• Machine learning (ML) models demonstrate significant potential in healthcare, particularly in Intensive Care Units (ICUs).
• Despite this potential, the practical application of ML models in ICUs remains limited, indicating the presence of significant, yet poorly understood, barriers to their adoption.
• This study aims to identify and characterize these barriers to ML implementation in critical care environments.

Purpose of the Study:

• To identify the barriers preventing the widespread adoption of machine learning models in Intensive Care Units (ICUs).
• To quantify the impact of identified technical challenges on the performance of predictive models in critical care.

Main Methods:

• A qualitative study involving 29 interviews with 40 staff members from ICUs, hospitals, and MedTech companies was conducted.
• Quantitative experiments were performed using two selected ML models to measure performance degradation due to data drift, feature changes, data scarcity, and context transfer.
• The study combined qualitative insights into barriers with quantitative validation of their impact on model performance.

Main Results:

• The qualitative study confirmed the potential of AI-driven analytics in patient care and highlighted prevalent technical obstacles hindering ML adoption in ICUs.
• Experimental results demonstrated that factors such as data drift, changing patient features, data scarcity, and deployment context significantly reduce predictive model performance.
• Each identified technical issue was shown to negatively impact ML model performance, with the extent of the impact varying by model and specific issue.

Conclusions:

• Further research is essential to develop practical solutions for enabling AI-driven innovation within Intensive Care Units.
• The limited availability of public, usable implementations of predictive models hinders scientific reproducibility and the translation of research into clinical practice.
• Addressing technical barriers and improving model implementation are crucial for advancing the use of AI in critical care medicine.