Natural Language Processing for Enhanced Clinical Decision Support in Allergy Verification for Medication...

Area of Science:

• Natural Language Processing
• Clinical Informatics
• Pharmacovigilance

Background:

• Electronic health records (EHRs) contain valuable patient data but are often unstructured.
• Extracting specific information like allergy details from Spanish EHRs is challenging.
• Automated methods are needed to improve data extraction and patient safety.

Purpose of the Study:

• To develop and validate a BERT-based named entity recognition (NER) model for extracting allergy information from Spanish EHRs.
• To assess the model's performance in identifying medication names, adverse reactions, and prescription errors.
• To evaluate the model's utility in clinical decision support for enhancing medication safety.

Main Methods:

• A BERT-based NER model was fine-tuned on 16,176 manually annotated allergy-related entities from Spanish EHRs.
• The dataset was split into training (80%) and testing (20%) subsets.
• Model performance was evaluated using accuracy, recall, F1 score, sensitivity, specificity, and Cohen κ, with expert review as the gold standard.

Main Results:

• The model achieved an overall F1 score of 0.80, with high performance for medication names (F1=0.91) and adverse reactions (F1=0.85).
• It demonstrated high specificity (99.98%) in identifying non-allergic cases and detected prescription errors in 0.96% of cases.
• The model showed substantial agreement with expert annotations (weighted κ=0.7797).

Conclusions:

• The Spanish-language BERT-based NER model shows strong performance in extracting allergy-related information and identifying non-allergic cases.
• The model holds promise for clinical decision support, particularly in enhancing medication safety despite moderate sensitivity.
• This study demonstrates the feasibility of using NER models for improving medication safety in Spanish-speaking healthcare settings.