Detection of Youth Suicide Interventions in Clinical Record Text using an Open-Source Language Model

This study presents an automated approach to detect youth suicide prevention interventions documented in emergency department notes. Expert review classified four interventions across 1,794 notes from 200 emergency visits for suicidality among children aged 6-17. The open-source language model Llama3.3-70B generated Likert scores (-3 to +3) for intervention presence. Scores approximated human classification: Lethal Means Restriction (AUROC 0.972, 95%CI:0.961-0.982), Hotline (AUROC 0.980, 95%CI:0.966-0.990), Outpatient Referral (AUROC 0.935, 95%CI:0.922-0.947), Safety Planning (AUROC 0.954, 95%CI:0.930-0.975). Application to 6,687 notes from 723 encounters revealed increased odds of omitted interventions among youth with prior emergency visits (OR 0.454-0.625), missing screening questions (OR 0.169-0.653), and ideation (vs. acts or attempts) (OR 0.296, 95%CI 0.133-0.657). Findings demonstrate systematic clinical text analysis with an open-source language model can expose new targets to inform decision support and strengthen evidence-based care for youth suicide.