Adversarial Multi-Teacher Distillation for Semi-Supervised Relation Extraction.

Area of Science:

• Natural Language Processing
• Machine Learning

Background:

• Labeled data scarcity is a major hurdle for relation extraction (RE).
• Semi-supervised RE (SSRE) uses pseudolabels from unlabeled data, but these can be erroneous.
• Erroneous pseudolabels introduce misleading information into SSRE models.

Purpose of the Study:

• To propose a novel framework, adversarial multi-teacher distillation (AMTD), for more effective knowledge capture in SSRE.
• To mitigate the negative impact of noisy pseudolabels in semi-supervised learning.

Main Methods:

• Developed a general knowledge distillation (KD) technique incorporating pseudolabels and prediction distributions from multiple models.
• Integrated a language model-based adversarial training (AT) component to enhance distillation robustness.
• Applied the AMTD framework to existing SSRE methods.

Main Results:

• The proposed AMTD framework significantly improved the performance of base SSRE methods.
• Experiments were conducted on two public datasets, demonstrating the framework's effectiveness.
• The refined knowledge capture through KD and AT led to superior relation extraction outcomes.

Conclusions:

• The AMTD framework offers a robust solution to the challenge of noisy pseudolabels in SSRE.
• This approach effectively refines knowledge extraction from unlabeled data, enhancing model accuracy.
• Adversarial multi-teacher distillation represents a significant advancement for semi-supervised relation extraction tasks.