Implementation of an opioid use disorder (OUD) machine-learning phenotype in real-time for the ADAPT clinical trial

We developed and deploy a real‑time, electronic health record‑integrated machine learning phenotype to identify emergency department patients with opioid use disorder for prospective clinical trial screening and buprenorphine initiation. We conducted a multi‑phase study across three emergency departments in a single United States health system from 2014 to 2025. Using visit‑level data available at or before triage, we trained a random‑forest classifier to estimate opioid use disorder risk and embedded scoring in the electronic health record to trigger point‑of‑care alerts for trial eligibility review. A high-specificity computable training label supported retrospective development, and a clinician gold-standard reference was established through structured chart review. Performance was summarized using discrimination, calibration, and threshold-based classification metrics; prospective validation used a stratified random sample of flagged and unflagged encounters. Retrospective discrimination compared to the high-specificity computable training label was high (area under the receiver operating characteristic curve, 0.99; 95% CI, 0.98-0.99; area under the precision-recall curve, 0.92; 95% CI, 0.90-0.94), and calibration plots informed operating‑point selection for real‑time use. In prospective gold‑standard validation (n = 217), 89.3% of model-positive encounters and 95.7% of model-negative encounters agreed with physician adjudication at the prespecified threshold. Because the sample was stratified on model prediction, these are within-stratum agreement estimates; weighting them by the source-population flag rate (28,284 of 866,569 eligible encounters, 3.26%) yields design-weighted estimates of population-level prevalence (7.1%), sensitivity (0.40, 95% CI 0.24-0.64), and specificity (0.996). An electronic health record‑embedded, machine learning phenotype can accurately and feasibly identify emergency department patients with opioid use disorder in real time, streamlining clinical trial enrollment and treatment initiation. Ongoing work will report operational metrics (e.g., alert volume and latency), monitor performance drift and equity across subgroups, and evaluate downstream clinical and trial outcomes.