A local outpatient practice-level prediction model for short-term psychiatric emergency presentation

To determine whether statewide health information exchange (HIE) data can support practice-level prediction of short-term psychiatric emergency presentation, and to compare a locally trained model with general-purpose clinical risk scores.

Retrospective cohort of patients at 1 Montana psychiatry practice with records in the Big Sky Care Connect statewide HIE. Structured HIE data were linked to zip code-level Social Vulnerability Index and Area Deprivation Index measures. Each observation was a patient snapshot. Logistic regression, random forest, and LightGBM models were tuned with 5 × 5 nested cross-validation and evaluated on a temporally held-out test set. The primary outcome was psychiatric or substance-related emergency department (ED) presentation within 30 days. Discrimination was compared with a parsimonious clinical baseline and the LACE and Elixhauser indices.

The test set contained 16 469 snapshots, of which 343 (2.1%) preceded a qualifying ED presentation (102 visits among 68 patients). A random forest model achieved area under the receiver-operator characteristic curve 0.85 (95% CI, 0.75-0.91). At a threshold targeting one-third recall, precision was 0.29 (0.12-0.44) and number needed to evaluate 3.4 (2.3-8.1). Discrimination exceeded the clinical baseline (Δ area under the curve 0.10), LACE (0.19), and Elixhauser (0.27); all P values less than  .001.

Structured HIE data, without clinical notes, supported a practice-specific model that outperformed general-purpose scores such as LACE and Elixhauser. Wide CIs reflect a small event count and constrain point-estimate claims.

health information exchange infrastructure may allow participating practices to build locally calibrated psychiatric risk models. Prospective study of clinical utility is needed.