Bleeding Entity Recognition in Electronic Health Records: A Comprehensive Analysis of End-to-End Systems

Area of Science:

• Biomedical Informatics
• Clinical Data Science
• Natural Language Processing

Background:

• Bleeding events are common adverse drug reactions in patients receiving anticoagulation therapy.
• Accurate identification of bleeding events is crucial for managing anticoagulation in atrial fibrillation.
• Electronic health records (EHR) often lack uniform capture of bleeding events, necessitating efficient extraction methods.

Purpose of the Study:

• To evaluate the effectiveness of various natural language processing (NLP) methods for automatic extraction of bleeding events from EHR data.
• To compare the performance of different NLP models, including biLSTM-CRF and BERT variants, in identifying bleeding events.

Main Methods:

• Utilized a dataset of 1,079 expert-annotated, de-identified EHR notes.
• Evaluated state-of-the-art NLP models, specifically biLSTM-CRF with language modeling and various BERT architectures.
• Assessed model performance for six distinct entity types related to bleeding events.

Main Results:

• The biLSTM-CRF model achieved the highest performance for entity-level extraction with a macro F1-score of 0.75.
• Sentence-level and document-level predictions demonstrated strong performance, with macro F1-scores of 0.84 and 0.96, respectively.
• Error analysis identified challenges including variable entity spans, model memorization, and difficulties with negation signals.

Conclusions:

• Natural language processing offers a viable solution for automated bleeding event extraction from EHRs.
• NLP models show promise in improving the capture of adverse drug reactions for anticoagulated patients.
• Further refinement of NLP models is needed to address specific error patterns for enhanced clinical data analysis.