Forecasting emergency department overcrowding using the CEDOCS score - predictable, but not actionable?

This study aims to develop an explainable machine learning (ML) model to predict emergency department (ED) overcrowding using the CEDOCS score and evaluate its perceived actionability in clinical practice.

We retrospectively analyzed all 300,000 visits to two German EDs between 2019 and 2023. We derived electronic health records (EHR), calendar and ambulance features to predict the CEDOCS score in the next three hours. We used SHAP values to analyze feature attributions. In a six-month intervention study, we introduced the ML model at one of the EDs and assessed perceived tool accuracy, actions taken based on the system's predictions, and stress.

The best-performing models achieved RMSEs of 11.76, 15.92, and 18.34 at Hospital 1 and 16.04, 21.38, and 22.91 at Hospital 2 for the one-, two- and three-hour forecasts, respectively. The current CEDOCS score, time of day, and ambulance diversion by the intensive care units had the strongest correlation with overcrowding. In the intervention study, clinician stress showed a small-to-moderate association with CEDOCS (r = 0.24, p = 0.03). Actions taken were unrelated to CEDOCS (p = 0.18) and model error (p = 0.21) but positively associated with perceived tool accuracy (r = 0.38, p < 0.01).

Our ML model demonstrates good predictive performance in forecasting ED overcrowding using the CEDOCS score. The SHAP analysis aligns with known causes of overcrowding. However, our intervention study suggests that accurate prediction of CEDOCS alone may be insufficient for actionable clinical decision support, as occupancy-based metrics may not fully capture the frontline resource demands that drive clinicians' operational decisions.