Identifying Rare Circumstances Preceding Female Firearm Suicides: Validating A Large Language Model Approach

Area of Science:

• Public Health
• Computer Science
• Criminology

Background:

• Female firearm suicide rates increased by 20% from 2010 to 2020, necessitating investigation into unique contributing factors.
• The National Violent Death Reporting System (NVDRS) contains valuable narrative data but conventional NLP methods struggle with identifying rare circumstances.
• Previous natural language processing (NLP) approaches were limited by insufficient data for identifying infrequent factors in female firearm suicides.

Purpose of the Study:

• To employ a large language model (LLM) to detect infrequent circumstances preceding female firearm suicides.
• To analyze unstructured narrative reports from the NVDRS using advanced AI techniques.
• To improve the identification of rare contributing factors in female firearm suicide incidents.

Main Methods:

• Utilized narrative reports from 1462 female firearm suicide decedents in the NVDRS (2014-2018).
• Coded 9 infrequent circumstances and trained an LLM to predict their presence in a yes/no format.
• Evaluated prediction accuracy using F1-scores, comparing LLM performance against a support vector machine (SVM).

Main Results:

• The LLM significantly outperformed the conventional SVM approach in identifying infrequent circumstances.
• LLM achieved F1-scores over 0.6 for 4 circumstances and 0.8 for 2 circumstances, compared to SVM's <0.2 for most.
• Demonstrated the capability of LLMs to extract nuanced information from unstructured text data.

Conclusions:

• LLM approaches show significant promise for analyzing complex narrative data in public health research.
• Researchers can leverage LLMs to identify rare circumstances in suicide data, potentially leading to more targeted prevention strategies.
• This study highlights the value of advanced AI in uncovering critical insights from previously underutilized data sources.