Developing a natural language processing system using transformer-based models for adverse drug event detection in...

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning in Healthcare
• Clinical Informatics

Background:

• Adverse Drug Events (ADEs) pose significant risks to patient safety.
• Accurate detection of ADEs from clinical notes is crucial for patient care.
• Existing methods for ADE detection often face challenges with unstructured EHR data.

Purpose of the Study:

• To develop and evaluate a transformer-based NLP system for ADE detection in electronic health records (EHRs).
• To compare different data processing strategies (window-based vs. split-based) for optimizing ADE detection performance.
• To assess the generalizability of the developed models on diverse EHR datasets.

Main Methods:

• Fine-tuning transformer models, including BERT Short-Formers and Clinical-Longformer, on clinical notes.
• Implementing and comparing window-based and split-based text processing approaches.
• Evaluating model performance using metrics like Area Under the Precision-Recall Curve (AUPRC) and F1 scores on n2c2 and VUMC datasets.

Main Results:

• Transformer models achieved high performance in ADE detection, with specific models and processing methods excelling on different datasets.
• Clinical-Longformer with split-based processing demonstrated strong results, particularly the 4-chunk split.
• Model generalizability was tested on Vanderbilt University Medical Center (VUMC) EHR data, showing varied performance based on processing methods.

Conclusions:

• Transformer-based NLP systems show significant potential for accurate ADE detection from clinical notes.
• Split-based data processing generally yielded better results than window-based methods for ADE detection.
• The choice of text preprocessing and model architecture significantly impacts performance and should consider note characteristics and annotation burden.