Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large...

Area of Science:

• Natural Language Processing (NLP)
• Biomedical Informatics

Background:

• Biomedical Named Entity Recognition (NER) is crucial for extracting information from biomedical texts.
• Large Language Models (LLMs) show potential for NER, especially in few-shot scenarios with limited data.

Purpose of the Study:

• To enhance LLM performance for few-shot biomedical NER.
• To investigate the effectiveness of dynamic prompting strategies, specifically retrieval-augmented generation (RAG).

Main Methods:

• Implemented and compared static and dynamic prompt engineering techniques for LLMs.
• Utilized retrieval-augmented generation (RAG) with TF-IDF and SBERT for dynamic example selection.
• Evaluated performance on five biomedical NER datasets across 5-shot and 10-shot settings.

Main Results:

• Static prompting with structured components improved F1-scores by 11-12% for GPT-4, GPT-3.5, and LLaMA 3-70B.
• Dynamic prompting further enhanced performance, with TF-IDF and SBERT retrieval yielding average F1-score improvements of 7.3% and 5.6% respectively.
• Dynamic RAG demonstrated superior results in few-shot biomedical NER tasks.

Conclusions:

• Contextually adaptive prompts via RAG are highly effective for improving few-shot biomedical NER.
• Dynamic prompting strategies offer a significant advancement over static methods for LLM-based biomedical NLP tasks.