Related Concept Videos
Improving Translational Accuracy
Improving Translational Accuracy
Language and Cognition
You might also read
Related Articles
Articles linked to this work by shared authors, journal, and citation graph.
A dynamic risk prediction framework for Alzheimer's disease and related dementias with interpretability.
Clinical document metadata extraction: A scoping review.
Context matching is not reasoning when performing generalized clinical evaluation of generative language models.
Related Experiment Video
Updated: Jan 18, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Large Language Models Struggle in Token-Level Clinical Named Entity Recognition.
Qiuhao Lu1, Rui Li1, Andrew Wen1
1McWilliams School of Biomedical Informatics, University of Texas Health Science Center, Houston, TX, USA.
Large Language Models (LLMs) show promise for rare disease Named Entity Recognition (NER). This study explores local and proprietary LLMs for token-level clinical NER, identifying challenges and improvements.
Area of Science:
- Healthcare Informatics
- Artificial Intelligence
- Clinical Natural Language Processing
Background:
- Large Language Models (LLMs) offer potential in healthcare, especially for rare diseases with data challenges.
- Named Entity Recognition (NER) is crucial for extracting clinical information, but current LLM research focuses on document-level NER.
- A gap exists in token-level clinical NER using local open-source LLMs.
Purpose of the Study:
- To investigate the effectiveness of proprietary and local LLMs for token-level clinical NER.
- To address the challenges of data scarcity and complexity in rare disease text analysis.
- To explore different LLM application methods including prompting and fine-tuning.
Main Methods:
- Experiments using zero-shot prompting, few-shot prompting, retrieval-augmented generation (RAG), and instruction-fine-tuning.
- Evaluation of both proprietary and local LLMs on token-level clinical NER tasks.
- Focus on clinical texts, particularly those related to rare diseases.
Main Results:
- LLMs face inherent challenges in token-level NER, especially within the rare disease domain.
- The study identifies specific difficulties LLMs encounter in precise clinical entity extraction.
- Insights into potential improvements for LLM application in healthcare NER were gained.
Conclusions:
- Token-level clinical NER using LLMs, particularly local models, presents significant challenges.
- Further research and model refinement are needed for effective LLM deployment in rare disease informatics.
- This study contributes to advancing LLM applications in specialized healthcare contexts.
