DocR-BERT: Document-Level R-BERT for Chemical-Induced Disease Relation Extraction via Gaussian Probability...

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Computational Biology

Background:

• Chemical-induced disease (CID) relation extraction is crucial for advancing disease treatment and drug development.
• Current methods struggle to capture complete document-level semantic information by overlooking inter-sentential entity relationships.
• Accurate extraction of CID relations from biomedical literature remains a significant challenge.

Purpose of the Study:

• To propose an effective document-level relation extraction model for automatically identifying intra- and inter-sentential CID relations.
• To enhance the capture of semantic information across different parts of a document (title, abstract, dependency paths).
• To improve the performance of CID relation extraction by considering document-wide context.

Main Methods:

• Utilized BERT for contextual semantic representations of title, abstract, and shortest dependency paths (SDPs).
• Introduced a cross2self-attention mechanism to learn mutual semantic information between document components.
• Employed Gaussian probability distribution to weight sentence importance and a document-level R-BERT (DocR-BERT) for comprehensive entity information.
• Concatenated representations and used a softmax function for CID extraction.

Main Results:

• The proposed DocR-BERT model achieved superior performance on the CDR corpus without external resources.
• Achieved F1-scores of 53.5% for inter-sentential, 70% for intra-sentential, and 63.7% overall.
• Demonstrated that cross2self-attention, Gaussian probability distribution, and DocR-BERT significantly enhance CID extraction.

Conclusions:

• The developed document-level model effectively extracts chemical-induced disease relations by integrating information across sentences.
• The proposed cross2self-attention mechanism and Gaussian weighting are key components for improving extraction accuracy.
• This approach offers a more complete understanding of CID relationships, benefiting biomedical research and drug discovery.