From zero to hero: Harnessing transformers for biomedical named entity recognition in zero- and few-shot contexts

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Machine Learning

Background:

• Supervised named entity recognition (NER) in biomedicine requires extensive annotated datasets, which are costly and time-consuming to create.
• Retraining models for new entities is inefficient, hindering the adaptability of current NER systems.

Purpose of the Study:

• To develop a zero- and few-shot NER method for the biomedical domain.
• To address the challenges of data annotation and model retraining for novel biomedical entities.

Main Methods:

• Transformed multi-class token classification into binary token classification.
• Pre-trained models on numerous datasets and biomedical entities to learn semantic relations.
• Utilized a fine-tuned PubMedBERT-based model.

Main Results:

• Achieved average F1 scores of 35.44% (zero-shot), 50.10% (one-shot), 69.94% (10-shot), and 79.51% (100-shot) across 9 diverse biomedical entities.
• Demonstrated superior performance compared to previous transformer-based methods.
• Showed comparable results to GPT3-based models with significantly fewer parameters.

Conclusions:

• The proposed method effectively recognizes new biomedical entities with limited or no examples.
• The approach offers a more efficient and scalable solution for biomedical NER.
• Publicly released models and code facilitate further research and application.