APLUS: A Python library for usefulness simulations of machine learning models in healthcare.

Area of Science:

• Medical Informatics
• Machine Learning in Healthcare
• Clinical Workflow Optimization

Background:

• Thousands of machine learning (ML) models exist, yet their clinical adoption for patient care improvement is limited.
• A key barrier is the disconnect between ML model evaluation and the practical requirements for clinical workflow integration.
• Existing methods for assessing ML model utility within clinical workflows are insufficient.

Purpose of the Study:

• To introduce APLUS, a reusable framework for quantitatively assessing the utility of integrating ML models into clinical workflows via simulation.
• To address the limitations in evaluating ML model usefulness within the context of care delivery.

Main Methods:

• Development of the APLUS simulation engine.
• Creation of a workflow specification language for APLUS.
• Application of APLUS to evaluate an ML-based peripheral artery disease screening pathway at Stanford Health Care.

Main Results:

• The APLUS framework provides a quantitative method for assessing ML model utility in clinical settings.
• Simulation results demonstrate the potential of APLUS in evaluating novel ML-driven clinical pathways.
• The study successfully applied APLUS to a real-world peripheral artery disease screening scenario.

Conclusions:

• APLUS offers a novel solution to bridge the gap between ML model development and clinical implementation.
• Quantitative simulation of ML model integration into workflows is crucial for realizing the promise of ML in healthcare.
• This framework facilitates the assessment of ML model impact on patient care delivery.