Biomedical named entity recognition based on multi-cross attention feature fusion.

Area of Science:

• Biomedical Natural Language Processing
• Computational Biology
• Bioinformatics

Background:

• Current biomedical named entity recognition (NER) methods often use character-level models like Character-level Convolutional Neural Networks (CharCNN) or Character-level Recurrent Neural Networks (CharRNN) independently.
• This independent usage overlooks the complementary strengths of these models and ignores crucial feature information during word integration, limiting overall performance.

Purpose of the Study:

• To propose a novel multi-cross attention feature fusion method for enhancing biomedical named entity recognition.
• To effectively integrate character-level and word-level features by leveraging cross-attention mechanisms.

Main Methods:

• The proposed method employs DistilBioBERT, CharCNN, and CharLSTM for initial cross-attention fusion between word and character features.
• A subsequent cross-attention fusion step integrates these initial feature vectors to create a final feature representation.
• A BiLSTM with a multi-head attention mechanism is utilized to refine feature focus and enhance performance.

Main Results:

• The model achieved top F1-scores across five benchmark datasets: NCBI-Disease (90.76%), BC5CDR-Disease (89.79%), BC5CDR-Chem (94.98%), JNLPBA (80.27%), and BC2GM (88.84%).
• These results demonstrate superior performance compared to existing methods.

Conclusions:

• The multi-cross attention feature fusion approach effectively captures richer semantic features.
• The proposed model significantly improves the accuracy and capability of biomedical named entity recognition.