Biomedical named entity recognition based on multi-cross attention feature fusion

Dequan Zheng ^1,2, Rong Han ¹, Feng Yu ^1,2

⁰Harbin University of Commerce, Harbin, China.

Plos One +

|

May 28, 2024

View abstract on PubMed

Summary

This study introduces a novel multi-cross attention fusion method for biomedical named entity recognition. The approach enhances feature extraction by integrating character and word-level information, significantly improving entity recognition accuracy.

Area Of Science

Biomedical Natural Language Processing
Computational Biology
Bioinformatics

Background

Current biomedical named entity recognition (NER) methods often use character-level models like Character-level Convolutional Neural Networks (CharCNN) or Character-level Recurrent Neural Networks (CharRNN) independently.
This independent usage overlooks the complementary strengths of these models and ignores crucial feature information during word integration, limiting overall performance.

Purpose Of The Study

To propose a novel multi-cross attention feature fusion method for enhancing biomedical named entity recognition.
To effectively integrate character-level and word-level features by leveraging cross-attention mechanisms.

Main Methods

The proposed method employs DistilBioBERT, CharCNN, and CharLSTM for initial cross-attention fusion between word and character features.
A subsequent cross-attention fusion step integrates these initial feature vectors to create a final feature representation.
A BiLSTM with a multi-head attention mechanism is utilized to refine feature focus and enhance performance.

Main Results

The model achieved top F1-scores across five benchmark datasets: NCBI-Disease (90.76%), BC5CDR-Disease (89.79%), BC5CDR-Chem (94.98%), JNLPBA (80.27%), and BC2GM (88.84%).
These results demonstrate superior performance compared to existing methods.

Conclusions

The multi-cross attention feature fusion approach effectively captures richer semantic features.
The proposed model significantly improves the accuracy and capability of biomedical named entity recognition.