How Do Your Biomedical Named Entity Recognition Models Generalize to Novel Entities?

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Computational Biology

Background:

• The rapid growth of biomedical literature necessitates robust biomedical named entity recognition (BioNER) for identifying novel concepts.
• Existing BioNER models may struggle to generalize to new and unseen entity mentions, despite high overall performance on benchmarks.

Purpose of the Study:

• To systematically analyze the generalization capabilities (memorization, synonym, and concept generalization) of BioNER models.
• To identify limitations and failure cases of current BioNER models in recognizing unseen biomedical entities.
• To investigate the impact of dataset biases and morphological patterns on model generalization.

Main Methods:

• Systematic analysis of BioNER model performance across memorization, synonym generalization, and concept generalization.
• Investigation of model failure cases, focusing on dataset biases and novel morphological patterns in biomedical names.
• Application of a statistics-based debiasing method to improve generalization to unseen mentions.

Main Results:

• Current state-of-the-art BioNER models exhibit limitations in synonym and concept generalization, suggesting their capabilities are overestimated.
• Models tend to exploit dataset biases, hindering generalization.
• Novel morphological patterns in biomedical names pose challenges for recognition.
• A debiasing method demonstrated improvement in generalizing to unseen mentions.

Conclusions:

• BioNER models require enhanced generalization capabilities to reliably identify new biomedical concepts in rapidly expanding literature.
• Addressing dataset biases and developing methods to handle novel name patterns are crucial for improving BioNER reliability.
• Further research into BioNER generalization is essential for advancing biomedical text mining and knowledge discovery.