How Well Do AI-Enabled Decision Support Systems Perform in Clinical Settings?

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support Systems

Background:

• Machine learning (ML) models are increasingly integrated into clinical decision support (CDS) systems.
• Evaluating the real-world performance of these ML-based CDS tools is critical for safe and effective clinical implementation.
• Existing evidence on the performance of contemporary ML-based CDS in actual clinical settings requires systematic examination.

Purpose of the Study:

• To systematically review and analyze the real-world performance of machine learning-based clinical decision support systems.
• To identify common ML tasks, methods, and performance metrics reported in the literature.
• To assess the gap between model validation performance and real-world clinical performance.

Main Methods:

• A systematic literature search was conducted across four bibliographic databases.
• Studies published over a 5-year period were included.
• Data extracted included CDS task, ML type and method, and reported real-world performance metrics.

Main Results:

• 32 studies were identified, focusing primarily on image recognition/interpretation (38%) and risk assessment (28%).
• Supervised learning (88%) was the predominant ML approach, with random forests and convolutional neural networks being common methods.
• Only 12 studies reported real-world performance, with significant performance degradation observed in clinical settings compared to initial validation.

Conclusions:

• Real-world performance of ML-based CDS often falls short of validation benchmarks.
• There is a critical need for standardized and comprehensive reporting of ML model performance in clinical settings.
• Enhancing reporting practices is fundamental to ensuring the safe and effective deployment of ML in healthcare.