Comparison of 2 Natural Language Processing Methods for Identification of Bleeding Among Critically Ill Patients

Area of Science:

• Clinical informatics
• Natural Language Processing (NLP)
• Patient Safety

Background:

• Adverse events in large patient populations are difficult to detect using structured data alone.
• Clinical notes contain valuable information about patient events but are challenging to analyze at scale.
• Automated methods are needed to improve the identification of adverse events for enhanced patient safety.

Purpose of the Study:

• To develop and compare two NLP methods: a rules-based approach and a machine learning (ML) approach.
• To evaluate the effectiveness of these methods in identifying bleeding events within clinical notes.
• To determine which NLP approach offers superior performance for detecting adverse events.

Main Methods:

• A diagnostic study utilized deidentified clinical notes from the Medical Information Mart for Intensive Care (MIMIC) database (2001-2012).
• A rules-based NLP approach was developed using a bleeding-specific dictionary.
• Three ML models (support vector machine, extra trees, convolutional neural network) were trained and compared against the rules-based approach.
• Notes were represented using term frequency-inverse document frequency (TF-IDF) and global vectors for word representation (GloVe).

Main Results:

• The rules-based approach demonstrated high sensitivity (91.1%) and specificity (84.6%).
• The rules-based model achieved a positive predictive value of 62.7% and a negative predictive value of 97.1%.
• While ML models showed comparable sensitivity, the rules-based approach exhibited better overall performance, particularly in specificity and negative predictive value.

Conclusions:

• Automated NLP methods can reliably detect adverse events like bleeding in clinical notes.
• The rules-based NLP approach proved more effective than ML methods for identifying bleeding events.
• This study highlights the potential of NLP for scalable adverse event detection, contributing to improved patient safety.