Vaner2: towards more general biomedical named entity recognition using multi-task large language model encoders.

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Machine Learning

Background:

• Biomedical named entity recognition (BioNER) is crucial for downstream tasks like relation extraction and clinical text analysis.
• BERT-based models, while dominant, struggle with generalization across different BioNER datasets.
• Existing fine-tuned autoregressive large language models (LLMs) are not optimal for BioNER, limiting performance.

Purpose of the Study:

• To develop a novel BioNER model that overcomes the generalization limitations of previous approaches.
• To leverage LLMs effectively for sequence labeling tasks in the biomedical domain.
• To improve the performance and adaptability of BioNER systems.

Main Methods:

• Utilized LLMs with the causal attention mask removed as a text encoder for sequence labeling.
• Trained a multi-task Named Entity Recognition (NER) model on 39 BioNER datasets for comprehensive entity extraction.
• Proposed a token-wise loss rescaling technique to address data imbalance issues between tags and entity types.

Main Results:

• The VANER2 model demonstrated superior generalization capabilities on independent test datasets.
• Achieved state-of-the-art performance compared to BERT-based baselines and recent BioNER methods.
• Successfully extracted all entity types in a single LLM forward pass.

Conclusions:

• VANER2 offers enhanced generalization for BioNER tasks by effectively adapting LLMs.
• The proposed methods, including causal attention mask removal and loss rescaling, significantly improve BioNER performance.
• VANER2 provides a robust and adaptable solution for biomedical text analysis.