Do syntactic trees enhance Bidirectional Encoder Representations from Transformers (BERT) models for chemical-drug...

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Computational Biology

Background:

• Collecting chemical-drug relations is vital for biomedical research.
• Pre-trained transformer models like BERT have limitations on biomedical texts due to scarce annotated data for relation extraction.
• Syntactic information may enhance performance in chemical-drug relation extraction.

Purpose of the Study:

• To investigate if enriching BioBERT with syntactic information improves chemical-drug relation extraction.
• To propose and evaluate novel syntax-enhanced BioBERT models.

Main Methods:

• Developed three syntax-enhanced BioBERT models: Chunking-Enhanced-BioBERT, Constituency-Tree-BioBERT, and Multi-Task-Syntactic (MTS)-BioBERT.
• Tested an existing Late-Fusion model and ensemble systems combining syntax-enhanced and non-syntax-enhanced models.
• Conducted experiments on the BioCreative VII DrugProt corpus.

Main Results:

• Syntax-enhanced models generally degraded BioBERT's performance in biomedical relation extraction.
• Performance improved when the subject-object distance of candidate semantic relations was long.
• The impact of dependency parse quality on performance was explored.

Conclusions:

• Syntactic enrichment offers conditional benefits for chemical-drug relation extraction, particularly for long-distance relations.
• Further research is needed to optimize the integration of syntactic information into transformer models for biomedical text.
• The study provides insights into the effectiveness of different syntax-enhancement strategies.