Distant supervision for treatment relation extraction by leveraging MeSH subheadings.

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Computational Biology

Background:

• Biomedical literature is expanding rapidly, necessitating automated information extraction.
• Relation extraction is crucial for understanding semantic links between biomedical entities.
• Existing distant supervision methods for treatment relations require improvement.

Purpose of the Study:

• To develop a novel distant supervision approach for extracting binary treatment relationships.
• To generate high-quality positive and negative training examples from PubMed abstracts.
• To improve the performance and robustness of relation extraction models.

Main Methods:

• Proposed a novel distant supervision approach leveraging MeSH subheadings to generate training data from PubMed abstracts.
• Assessed training data quality by evaluating the performance of induced supervised models on a gold standard test set.
• Utilized bootstrapped ensembling for model training and evaluation.
• Investigated the augmentation of crowd-sourced datasets with generated examples.
• Implemented a classification loss resistant to label noise.

Main Results:

• The novel distant supervision approach achieved 81.38% PR-AUC for treatment relations, outperforming traditional distant supervision (64.33%).
• Generated data quality is closer to human crowd annotations (90.57% PR-AUC).
• Augmenting crowd-sourced datasets with generated examples improved model performance by over two absolute points on the F1 metric.
• A noise-resistant classification loss further enhanced model performance.

Conclusions:

• The proposed distant supervision method effectively generates high-quality training data for biomedical relation extraction.
• This approach offers a viable alternative to traditional distant supervision and complements human annotation efforts.
• The method demonstrates significant improvements in model performance and robustness, particularly for treatment relations.