A Machine Learning Strategy to Predict the Number of High-Acuity Children Who Leave Without Being Seen From the...

The purpose of this study was to create an operationally useful machine learning model that predicts the number of high-acuity left without being seen (LWBS) patients from the pediatric emergency department (ED).

We analyzed data from 512,616 ED visits between 2018 and 2024. We developed 6 models (univariate logistic regression using ED census, least absolute shrinkage and selection operator regularization, support vector machines, random forest, gradient boosting (XGBoost), and temporal fusion transformers) and a majority-vote ensemble to predict the number of high-acuity LWBS pediatric ED patients over an 8-hour period. We used 10 iterations of time series data for model development. Each iteration was trained on progressively increasing amounts of data from 2018 to 2023, using the subsequent 1-year period after training as the test set. The final iteration used all data from 2018 to 2023 for training and the entirety of 2024 for testing. Models were developed for both rolling 8-hour windows and the 12:00 PM to 8:00 PM timeframe, which was an operationalizable example based on our own institution's staffing schedule.

At the prespecified threshold of 2 or more high-acuity LWBS patients over 8-hour windows, the XGBoost and temporal fusion transformer models demonstrated the highest predictive performance with an area under the receiving operator curve of 0.85 (95% CI, 0.84-0.86) and 0.86 (95% CI, 0.84-0.87), respectively. These models also had the highest predictive performance for the daily 12:00 PM to 8:00 PM timeframe. A majority-vote, 3-model ensemble did not result in improved performance.

Machine learning models show excellent predictive performance in identifying optimal periods to operationally target and potentially mitigate the high-acuity LWBS phenomenon.