An Explainable Artificial Intelligence Text Classifier for Suicidality Prediction in Youth Crisis Text Line Users:...

Area of Science:

• Artificial Intelligence
• Clinical Psychology
• Public Health

Background:

• Suicide is a major public health issue.
• Machine learning (ML) models show promise in identifying individuals at risk of suicidal ideation and behaviors (SIB).
• Pretrained large language models (LLMs) have demonstrated effectiveness in analyzing speech and text for SIB prediction.

Purpose of the Study:

• Develop and implement ML methods using transformer-based LLMs to predict SIBs in a real-world crisis helpline dataset.
• Evaluate and benchmark the developed ML model against traditional text classification methods.
• Train an explainable AI model to identify features associated with suicide risk.

Main Methods:

• Analysis of chat protocols from adolescents and young adults (14-25 years) at a German crisis helpline.
• Development of an ML model utilizing a transformer-based LLM architecture with long short-term memory layers.
• Prediction of suicidal ideation (SI) and advanced suicidal engagement (ASE) using composite Columbia-Suicide Severity Rating Scale scores.
• Comparison with a word-vector-based ML model and computation of performance metrics including discrimination, calibration, clinical utility, and explainability via Shapley Additive Explanations (SHAP).

Main Results:

• The transformer-based model achieved a macroaveraged AUC-ROC of 0.89 and accuracy of 0.79, outperforming the baseline model (AUC-ROC=0.77, accuracy=0.61).
• The model showed excellent prediction for nonsuicidal sessions (AUC-ROC=0.96) and good prediction for SI (AUC-ROC=0.85) and ASE (AUC-ROC=0.87).
• SHAP analysis identified self-reference, negation, low self-esteem expressions, and absolutist language as key risk indicators.

Conclusions:

• Neural networks leveraging LLM transfer learning can accurately detect SI and ASE in crisis chat data.
• Explainable AI models can reveal language features linked to SIBs, potentially supporting clinical decision-making.
• Future research should investigate multimodal inputs and temporal dynamics for enhanced suicide risk assessment.