Span-aware pre-trained network with deep information bottleneck for scientific entity relation extraction.

Area of Science:

• Natural Language Processing
• Bioinformatics
• Computational Linguistics

Background:

• Scientific entity and relation extraction are crucial for understanding scientific literature.
• Existing models struggle with scientific semantic dilution and isolated task information.
• This leads to challenges in representation learning and subtask performance.

Purpose of the Study:

• To propose a novel Span-aware Pre-trained network with deep Information Bottleneck (SpIB) for scientific entity and relation extraction.
• To address the challenges of scientific semantic dilution and information isolation among subtasks.
• To improve the coherence of scientific semantics and overall extraction performance.

Main Methods:

• SpIB utilizes a minimum span-based representation learning (SRL) module to disentangle task-irrelevant information.
• A relatedness-oriented task-relevant representation learning (TRL) module discovers relationships within task-relevant information.
• An information minimum-maximum strategy and a unified loss function optimize these representations.

Main Results:

• SpIB significantly outperforms state-of-the-art models on scientific datasets like SciERC, ADE, and BioRelEx.
• The model effectively minimizes task-irrelevant information while maximizing task-relevant information relatedness.
• Improved coherence in scientific semantics leads to enhanced performance across subtasks.

Conclusions:

• The SpIB model offers a superior approach to scientific entity and relation extraction.
• By disentangling information and enhancing semantic relatedness, SpIB achieves state-of-the-art results.
• The proposed method provides a robust framework for scientific text understanding.