Dependency parsing of biomedical text with BERT.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Bioinformatics

Background:

• Syntactic analysis (parsing) is vital for text mining.
• Universal Dependencies (UD) is the leading dependency parsing standard.
• Biomedical text parsing using UD remains understudied.

Purpose of the Study:

• To apply advanced neural dependency parsing to biomedical texts.
• To evaluate parsers fine-tuned on the CRAFT-SA dataset.
• To assess the impact of transfer learning with BERT models.

Main Methods:

• Utilized state-of-the-art neural dependency parsers (Turku Neural Parser, UDify).
• Fine-tuned parsers on the CRAFT-SA shared task dataset, adhering to UD conventions.
• Evaluated transfer learning using various BERT models, including biomedical-specific ones.

Main Results:

• Achieved substantial improvements in biomedical text parsing accuracy over prior work.
• Demonstrated the effectiveness of neural parsing technology for specialized domains.
• Confirmed that in-domain pre-trained transfer learning models are key for optimal performance.

Conclusions:

• Neural dependency parsing shows high accuracy for biomedical texts.
• Transfer learning with biomedical BERT models significantly enhances parsing performance.
• This work advances NLP applications in the biomedical domain.