Developing a Tool for Identifying Clinical Risk From Free-Text Clinical Records: Natural Language Processing Study.

Area of Science:

• Computational linguistics
• Clinical informatics
• Mental health research

Background:

• Electronic patient records (EPRs) are rich data sources underutilized in research.
• Natural Language Processing (NLP) has been applied to EPRs, but not extensively within third-sector organizations.
• Veteran mental health charities hold valuable EPR data for risk assessment.

Purpose of the Study:

• To develop an NLP-based risk identification tool for discerning high and low suicide risk.
• To utilize EPRs from a UK-based veteran mental health charity for tool development.
• To apply machine learning classification to veteran clinical notes.

Main Methods:

• Extracted 20,342 clinical notes from veteran EPRs.
• Utilized a 70/30 split for training and testing datasets.
• Developed and trained a binary classification framework (1=high risk, 0=low risk).

Main Results:

• A logistic regression classifier demonstrated the best performance.
• The tool achieved a mean positive predictive value of 0.74 and negative predictive value of 0.73.
• Evaluated performance metrics included sensitivity (0.75), F1-score (0.74), and accuracy via Youden index (0.73).

Conclusions:

• The NLP tool successfully identified suicide risk categories in veterans from clinical notes.
• The tool shows promise for enhancing veteran mental healthcare.
• Future research should explore the tool's ability to detect nuanced risk differences and its generalizability.