A Hybrid Language Framework for Ontology-Based Clinical Concept Extraction

Area of Science:

• Natural Language Processing
• Biomedical Informatics
• Artificial Intelligence

Background:

• Electronic Health Records (EHRs) contain valuable clinical information in unstructured narrative text.
• Extracting clinical concepts accurately from EHRs is crucial for downstream applications like clinical decision support and research.
• Existing methods often struggle with the complexity and nuances of clinical language.

Purpose of the Study:

• To develop and evaluate a hybrid ontology-based framework for clinical concept extraction from EHR discharge summaries.
• To compare the performance of large language models (LLMs) against traditional systems for this task.
• To assess the effectiveness of integrating NLP, semantic similarity, and biomedical terminologies.

Main Methods:

• A sequential NLP pipeline was developed, incorporating SparkNLP for initial processing, SentenceBERT for semantic similarity, and LLMs (LLaMA3-8B, Mistral-7B) for concept selection.
• UMLS REST API was used for normalization to Unified Medical Language System (UMLS) Concept Unique Identifiers (CUIs) and SNOMED CT.
• The framework was tested on MIMIC-III discharge summaries, with clinician-based evaluation of extracted concepts.

Main Results:

• LLaMA3-8B achieved the highest F1 score (0.77) and the lowest false positive rate (3.04%), outperforming Mistral-7B and cTAKES.
• LLMs demonstrated superior ability in handling clinical language complexities compared to rule-based systems.
• Mistral-7B offered faster processing for shorter notes, while LLaMA3-8B excelled in accuracy for detailed sections.

Conclusions:

• The proposed hybrid framework offers a flexible and context-aware approach to clinical concept extraction.
• LLMs integrated within this framework show significant promise for improving the accuracy and efficiency of clinical information extraction from EHRs.
• Future research should focus on full ontology mapping, assertion detection, and validation on diverse datasets.