Machine learning models to detect opioid misuse in emergency department patients at triage

Emergency department (ED) encounters represent valuable opportunities to initiate evidence-based treatments for patients with opioid misuse, but few receive such care. Universal manual screening has been proposed to improve patient identification but is uncommon due to its time and resource-intensive nature. We sought to determine the feasibility of identifying patients with opioid misuse at the time of ED triage using machine learning (ML).

We conducted a retrospective cohort study of 1123 ED encounters (September 2020 - March 2023) at a tertiary hospital. Encounters were enriched for opioid misuse, manually annotated, and chronologically split for training, validation, and testing. Candidate triage-time features included patient demographics, Emergency Severity Index, arrival time of day, chief complaint, comorbidities, and chronic medications. Model performance was evaluated using F1 score, area under the precision-recall curve (AUPRC), accuracy, recall, and AUROC. Post-hoc explainability analyses included SHapley Additive exPlanations (SHAP) and feature importance.

All models performed comparably to opioid-related diagnosis codes placed at any time during the encounter. Random Forest (F1 = 0.75 [95%CI 0.70-0.83], AUPRC = 0.88 [0.81-0.93], accuracy = 0.79 [0.70-0.83]) and Gradient Boosting (F1 = 0.77 [0.71-0.82], AUPRC = 0.89 [0.85-0.93], accuracy = 0.81 [0.720.84]) had among the highest F1 score and AUPRC but confidence intervals overlapped with other methods. Explainability analyses highlighted prior drug-use diagnosis codes, triage acuity, and age as top predictors.

ML classifiers leveraging routinely collected triage data offer a feasible and scalable alternative to manual screening in flagging opioid misuse before physician evaluation, potentially enabling early harm-reduction interventions. Prospective multi-site validation, calibration, and bias assessments are warranted.