Deep learning with language models improves named entity recognition for PharmaCoNER.

Area of Science:

• Biomedical Natural Language Processing
• Computational Linguistics
• Pharmacology

Background:

• Accurate recognition of pharmacological entities is crucial for drug discovery and biomedical knowledge extraction.
• Existing efforts primarily focus on English, leaving a gap in multilingual biomedical NER.
• The PharmaCoNER challenge addresses pharmacological entity recognition in Spanish biomedical texts.

Purpose of the Study:

• To explore and compare various BERT models for the PharmaCoNER task.
• To investigate the effectiveness of deep learning language models for Spanish biomedical NER.
• To leverage existing NLP resources for the PharmaCoNER challenge.

Main Methods:

• Utilized and compared representative BERT models.
• Applied deep learning with language models to the PharmaCoNER dataset.
• Investigated the impact of domain knowledge, WordPiece tokenization, and character case.

Main Results:

• Deep learning language models significantly improved performance on the PharmaCoNER dataset.
• Achieved state-of-the-art performance with a maximum F1-score of 92.01%.
• Demonstrated the effectiveness of BERT models for Spanish pharmacological NER.

Conclusions:

• Biomedical domain knowledge is more impactful than native language for BERT models on PharmaCoNER.
• WordPiece tokenization helps mitigate out-of-vocabulary limitations.
• Custom domain-specific vocabularies and character case considerations can further enhance model performance.