Predicting opportunities for improvement in trauma care using machine learning: a retrospective registry-based study
Jonatan Attergrim1,2, Kelvin Szolnoky3, Lovisa Strömmer4,5
1Department of Global Public Health, Karolinska Institutet, Stockholm, Sweden jonatan.attergrim@ki.se.
Objective:
To develop models to predict opportunities for improvement in trauma care and compare the performance of these models to the currently used audit filters.
Design:
Retrospective registry-based study.
Setting:
Single-centre, Scandinavian level one equivalent trauma centre.
Participants:
8220 adult trauma patients screened for opportunities for improvement between 2013 and 2022.
Primary And Secondary Outcome Measures:
Two machine learning models (logistic regression and XGBoost) and the currently used audit filters were compared. Internal validation by an expanding window approach with annual updates was used for model evaluation. Performance measured by discrimination, calibration, sensitivity and false positive rate of opportunities for improvement prediction.
Results:
A total of 8220 patients, with a mean age of 45 years, were analysed; 69% were men with a mean injury severity score of 12. Opportunities for improvement were identified in 496 (6%) patients. Both the logistic regression and XGBoost models were well-calibrated, with intercalibration indices of 0.02 and 0.02, respectively. The models demonstrated higher areas under the receiver operating characteristic curve (AUCs) (logistic regression: 0.71; XGBoost: 0.74). The XGBoost model had a lower false positive rate at a similar sensitivity (false positive rate: 0.63). The audit filters had an AUC of 0.62 and a false positive rate of 0.67.
Conclusions:
The logistic regression and XGBoost models outperformed audit filters in predicting opportunities for improvement among adult trauma patients and can potentially be used to improve systems for selecting patients for trauma peer review.
