Expert-augmented machine learning for predicting extubation readiness in the pediatric intensive care unit.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Pediatric Critical Care

Background:

• Predicting extubation readiness in pediatric intensive care units (PICU) presents significant clinical challenges.
• Machine learning models offer potential but often lack generalizability.
• Integrating human expertise is crucial for refining predictive accuracy.

Purpose of the Study:

• To develop and evaluate an expert-augmented machine learning (EAML) model for predicting successful extubation in PICU patients.
• To assess the generalizability of the EAML model compared to a standard machine learning approach.
• To leverage clinician expertise to enhance the performance of predictive algorithms.

Main Methods:

• Utilized electronic health record data from two PICUs, splitting one for training/internal testing and the other for external validation.
• Employed RuleFit to generate decision rules from gradient-boosted trees.
• Incorporated expert clinician feedback to regularize the RuleFit model, creating the EAML model.
• Compared the performance of RuleFit and EAML using Area Under the Curve (AUC) metrics.

Main Results:

• The RuleFit model identified 46 rules, with feedback from 25 experienced PICU clinicians.
• In the internal test set, EAML showed a slight, non-significant decrease in performance (AUC 0.814) compared to RuleFit (AUC 0.817).
• Crucially, EAML demonstrated superior performance in the external test set (AUC 0.799) compared to RuleFit (AUC 0.791), indicating improved generalizability.

Conclusions:

• Expert-augmented machine learning (EAML) effectively integrates clinical expertise into predictive models.
• The EAML approach enhances the generalizability of models for predicting extubation success in PICU settings.
• Directly incorporating expert knowledge during model construction yields more robust and externally valid predictive tools.