BERT-GT: cross-sentence n-ary relation extraction with BERT and Graph Transformer

Area of Science:

• Biomedical Natural Language Processing
• Computational Biology
• Bioinformatics

Background:

• Biomedical relation extraction is crucial for understanding complex biological information.
• Existing methods often use graph neural networks (GNNs) or attention mechanisms for cross-sentence relation extraction.
• Transformers have recently shown superior performance in various natural language processing (NLP) tasks.

Purpose of the Study:

• To develop a novel architecture for biomedical relation extraction that enhances performance on long texts.
• To improve the accuracy and efficiency of identifying relationships between biomedical entities across multiple sentences.

Main Methods:

• Proposed a novel architecture, BERT-GT, integrating Bidirectional Encoder Representations from Transformers (BERT) with a Graph Transformer.
• Introduced a neighbor-attention mechanism within BERT, allowing tokens to focus on local context.
• Applied the BERT-GT model to cross-sentence and abstract-level relation-extraction tasks.

Main Results:

• Achieved significant improvements in accuracy (5.44%) and F1-measure (3.89%) over state-of-the-art methods.
• Demonstrated the effectiveness of the neighbor-attention mechanism for handling long biomedical texts.
• Validated the robustness and applicability of BERT-GT on n-ary and chemical-protein relation datasets.

Conclusions:

• BERT-GT offers a robust and effective approach for biomedical relation extraction.
• The neighbor-attention mechanism is critical for improving performance on tasks involving extensive text.
• The proposed model shows promise for broader applications in biomedical text mining and relation extraction.