Classifying Firearm Injury Intent in Electronic Hospital Records Using Natural Language Processing.

Area of Science:

• Medical Informatics
• Public Health
• Computational Medicine

Background:

• International Classification of Diseases (ICD)-coded hospital discharge data often misclassify the intent behind firearm injuries (e.g., assault, unintentional, self-harm).
• Accurate intent classification is crucial for understanding injury patterns and informing public health interventions.
• Electronic health record (EHR) narrative text contains rich contextual information that may improve intent classification.

Purpose of the Study:

• To evaluate the accuracy of a machine learning (ML) model in classifying firearm injury intent using natural language processing (NLP) on EHR narrative data.
• To compare the performance of the NLP-ML model against traditional ICD coding by medical record coders.

Main Methods:

• A retrospective review of EHR data from three level I trauma centers (Boston, MA, and Seattle, WA) between 2000-2019.
• Development and validation of a gradient-boosting classifier using NLP to extract intent-relevant features from narrative text.
• Comparison of model-assigned intent with ICD codes and expert-assigned intent, including external validation.

Main Results:

• The NLP-ML model demonstrated higher accuracy than ICD coding in classifying firearm injury intent at both the development and external validation sites.
• The model achieved notable F-scores for accident (0.78 vs. 0.40) and assault (0.90 vs. 0.78) intents at the development site.
• Performance was maintained on external data (accident F-score, 0.64 vs. 0.58; assault F-score, 0.88 vs. 0.81), with further improvement after retraining.

Conclusions:

• NLP and ML techniques applied to EHR narratives significantly enhance the accuracy of firearm injury intent classification compared to ICD codes.
• This improved accuracy is particularly relevant for prevalent and often misclassified intents like accidental and assault-related injuries.
• Future research should focus on refining these models with larger, more diverse datasets to further enhance their clinical and public health utility.