Natural Language Processing and ICD-10 Coding for Detecting Bleeding Events in Discharge Summaries: Comparative...

Area of Science:

• Medical Informatics
• Clinical Natural Language Processing
• Patient Safety

Background:

• Bleeding adverse drug events (ADEs) in older inpatients on antithrombotic therapy are a major hospital safety concern.
• Conventional rule-based systems using ICD-10 codes often fail to detect these events due to limited data granularity.

Purpose of the Study:

• To develop and evaluate a natural language processing (NLP) model for detecting and categorizing bleeding ADEs in older adults' discharge summaries.
• To compare the NLP model's performance against a rule-based algorithm using ICD-10 codes.

Main Methods:

• Manual annotation of 400 discharge summaries into "no bleeding," "clinically significant bleeding," "severe bleeding," and "history of bleeding."
• Development of an NLP model (logistic regression, SVM) with class-weighting and a rule-based ICD-10 algorithm.
• Performance evaluation using accuracy, precision, recall, F1-score, and ROC curve analyses.

Main Results:

• The NLP model significantly outperformed the rule-based approach across all metrics (e.g., macro-average F1-score of 0.80).
• NLP achieved high precision for severe (0.92) and clinically significant bleeding (0.87), with an AUC of 0.94 for differentiating bleeding severity.
• The rule-based ICD-10 model showed poor recall (0.03) for severe bleeding, missing rare conditions.

Conclusions:

• NLP offers a significant advantage over ICD-10-based methods for detecting bleeding ADEs in electronic medical records.
• The NLP model effectively captures clinical nuances like severity and history, enhancing patient safety surveillance.
• Future work should involve multi-institutional validation and refining NLP temporal reasoning.