Does Provider Identity at Triage Improve Machine Learning Prediction of Hospital Admission? A Comparative Analysis of...

Area of Science:

• Emergency Medicine
• Data Science
• Clinical Informatics

Background:

• Machine learning (ML) models demonstrate high accuracy in predicting hospital admission from emergency department (ED) triage data.
• The impact of incorporating provider identity, as a proxy for practice variation, on ML prediction accuracy remains understudied.

Purpose of the Study:

• To compare the predictive performance of 10 supervised ML classifiers for hospital admission at ED triage, with and without provider identity.
• To characterize the reasoning of top-performing ML models using SHapley Additive exPlanations (SHAP).

Main Methods:

• A retrospective cohort study involving 186,094 ED visits for training and 58,151 for testing at an academic tertiary-care ED.
• Ten ML classifiers were trained using 23 triage features, both with and without the addition of provider identity.
• SHAP analysis was performed on the best-performing models (CatBoost and XGBoost).

Main Results:

• The overall hospital admission rate was approximately 31.3% in the training set and 31.7% in the test set.
• The highest baseline AUC achieved was 0.8906 with the CatBoost model; adding provider identity resulted in negligible AUC changes across all models.
• SHAP analysis identified ESI level, respiratory rate, temperature, complaint category, and age as primary predictors, with clinically intuitive relationships.

Conclusions:

• Provider identity does not significantly enhance ML-based prediction of hospital admission beyond established triage variables.
• Observed variations in provider admission rates are primarily attributed to patient case-mix differences rather than independent practice patterns.
• SHAP analysis provides transparent and clinically interpretable explanations for ML model predictions, suitable for clinical decision support.