Applying large language models to stratify suicide risk using narrative clinical notes.

Area of Science:

• Artificial Intelligence in Healthcare
• Public Health Surveillance
• Clinical Risk Prediction

Background:

• Suicide risk assessment post-hospital discharge is critical for patient safety.
• Existing risk stratification methods have limitations in accuracy and scope.
• Large language models (LLMs) offer potential for analyzing complex clinical data.

Purpose of the Study:

• To evaluate the efficacy of a large language model (LLM) in stratifying suicide risk among patients after hospital discharge.
• To compare LLM-based risk prediction against traditional sociodemographic and clinical factors.

Main Methods:

• Utilized a HIPAA-compliant LLM (gpt-4-1106-preview) on discharge summaries from 458,053 adult patients.
• Matched 1995 suicide/accident decedents with 5 controls based on key demographics and clinical variables.
• Applied Fine and Gray competing risk regression to analyze predicted vs. observed risk.

Main Results:

• LLM successfully stratified suicide and accidental death risk, with significant differences across risk quartiles (p < .001).
• Predicted risk was strongly associated with observed outcomes (adjusted HR 8.86).
• Higher estimated risks were observed in Black and Hispanic individuals compared to white individuals (p < .005).

Conclusions:

• Large language models can effectively stratify suicide risk post-hospital discharge.
• LLM-based risk prediction surpasses traditional methods using sociodemographic and clinical data alone.
• This AI approach holds promise for enhancing suicide prevention strategies in healthcare settings.