Jove
Visualize
Contact Us
  1. Home
  3. Research Domains
  5. Information And Computing Sciences
  7. Artificial Intelligence
  9. Natural Language Processing
  11. Efficient Medical Ner With Limited Data: Enhancing Llm Performance Through Annotation Guidelines.
  1. Home
  3. Research Domains
  5. Information And Computing Sciences
  7. Artificial Intelligence
  9. Natural Language Processing
  11. Efficient Medical Ner With Limited Data: Enhancing Llm Performance Through Annotation Guidelines.

Related Experiment Video

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

980

Efficient medical NER with limited data: Enhancing LLM performance through annotation guidelines.

Emiko Shinohara1, Yoshimasa Kawazoe1

  • 1Artificial Intelligence and Digital Twin in Healthcare, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan.

International Journal of Medical Informatics
|December 23, 2025

View abstract on PubMed

Summary
This summary is machine-generated.

Incorporating detailed annotation guidelines into prompts significantly improves few-shot learning for medical Named Entity Recognition (NER) using large language models (LLMs). This approach enhances recall and F1 scores, offering a practical solution for resource-limited NLP development.

Keywords:
Artificial intelligenceLarge language modelMedical informaticsNamed entity recognition

More Related Videos

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

3.1K
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

1.2K

Related Experiment Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

980
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

3.1K
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

1.2K

Related Concept Videos

JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Area of Science:

  • Natural Language Processing (NLP)
  • Artificial Intelligence (AI)
  • Computational Linguistics

Background:

  • Named Entity Recognition (NER) is crucial in medical NLP for identifying key clinical information.
  • Traditional NER methods require extensive annotated data, posing resource challenges.
  • Large Language Models (LLMs) offer innovative few-shot learning approaches for NER.

Purpose of the Study:

  • To evaluate the impact of annotation guidelines within LLM prompts on few-shot NER performance.
  • To assess this impact across diverse medical text corpora.

Main Methods:

  • Eight prompt patterns were designed, combining few-shot examples with varying annotation guideline complexity.
  • Performance was evaluated using three LLMs (GPT-4o, Claude 3.5 Sonnet, gpt-oss-120b) on three medical corpora (i2b2-2014, i2b2-2012, MedTxt-CR).
Natural language processing
  • Accuracy metrics included precision, recall, and F1 score, aligned with relevant shared tasks.

    • Main Results:

    • The inclusion of detailed annotation guidelines in few-shot prompts generally led to improvements in recall and F1 scores.
    • Specific prompt structures and guideline complexity influenced performance variations across LLMs and corpora.

    Conclusions:

    • Integrating annotation guidelines into LLM prompts is an effective strategy to boost NER performance, especially in few-shot scenarios.
    • This method provides a practical and efficient way to develop accurate medical NLP systems, particularly in environments with limited annotated data.
    • Annotation guidelines are vital for both evaluation and prompt engineering, optimizing LLM capabilities in specialized domains like medical NLP.