Improving few-shot named entity recognition for large language models using structured dynamic prompting with...

Area of Science:

• Computational linguistics
• Bioinformatics
• Artificial Intelligence

Background:

• Biomedical named entity recognition (NER) is crucial for extracting information from biomedical texts.
• Large language models (LLMs) show potential for few-shot NER but face performance challenges.
• Prompting strategies are key to optimizing LLM performance in specialized domains like biomedicine.

Purpose of the Study:

• To enhance few-shot biomedical NER performance using innovative prompting techniques.
• To investigate the effectiveness of retrieval-augmented generation for biomedical NER.
• To evaluate static and dynamic prompt engineering strategies across different LLMs.

Main Methods:

• Implemented and evaluated a structured multi-component static prompt and a dynamic prompt engineering technique.
• Dynamic prompting utilized retrieval of relevant in-context examples based on input texts (TF-IDF retrieval).
• Tested strategies on five biomedical NER datasets using GPT-4, GPT-3.5, LLaMA 3-70B, and GPT-OSS-120B.

Main Results:

• Structured static prompting improved average F1-scores by 11-12% compared to basic prompting.
• Dynamic prompting further boosted performance, with TF-IDF retrieval yielding an 8.8% (5-shot) and 6.3% (10-shot) improvement in F1-scores.
• Ablation studies showed strong performance with small retrieval pool sizes, indicating scalability.

Conclusions:

• Innovative prompting strategies, particularly dynamic retrieval-augmented generation, significantly advance few-shot biomedical NER.
• The proposed framework demonstrates high annotation efficiency and scalability for real-world applications.
• LLMs, when guided by effective prompting, offer a powerful solution for biomedical text analysis.