Suicide Phenotyping from Clinical Notes in Safety-Net Psychiatric Hospital Using Multi-Label Classification with...

Area of Science:

• Artificial Intelligence in Healthcare
• Clinical Natural Language Processing
• Mental Health Informatics

Background:

• Accurate identification of suicidal events is crucial for effective suicide prevention strategies in psychiatric care.
• Pre-trained language models (PLMs) show potential for extracting suicidality information from unstructured clinical text.
• Existing methods require evaluation for detecting co-occurring suicidal events.

Purpose of the Study:

• To evaluate the performance of BERT-based language models in identifying coexisting suicidal events from clinical narratives.
• To compare different fine-tuning strategies for detecting suicidal ideation, suicide attempts, exposure to suicide, and non-suicidal self-injury.
• To determine the optimal model and strategy for improving suicide risk assessment in high-acuity psychiatric settings.

Main Methods:

• Utilized 500 annotated psychiatric evaluation notes labeled for suicidal ideation (SI), suicide attempts (SA), exposure to suicide (ES), and non-suicidal self-injury (NSSI).
• Evaluated four BERT-based models (RoBERTa, MentalBERT, BioClinicalBERT, BERT) using two fine-tuning strategies: multiple single-label and single multi-label classification.
• Assessed model performance using accuracy and F1-scores.

Main Results:

• RoBERTa achieved the highest performance with binary relevance (accuracy=0.86, F1=0.78).
• MentalBERT (F1=0.74) and BioClinicalBERT (F1=0.72) also demonstrated significant performance.
• Fine-tuning RoBERTa with a single multi-label classifier further improved performance (accuracy=0.88, F1=0.81).

Conclusions:

• Domain-relevant pre-trained models, particularly RoBERTa, are effective for identifying complex suicidal events in clinical notes.
• The single multi-label classification strategy enhances both the efficiency and performance of suicide risk detection.
• These findings support the use of advanced NLP models for improved suicide precautions and care quality.