TaughtNet: Learning Multi-Task Biomedical Named Entity Recognition From Single-Task Teachers.

Area of Science:

• Computational biology
• Natural Language Processing
• Bioinformatics

Background:

• Deep learning models for Biomedical Named Entity Recognition (BioNER) are limited by the scarcity of comprehensive annotated datasets.
• Existing datasets often focus on single entity types (e.g., diseases or drugs), hindering the development of effective multi-task BioNER systems.
• Training multi-task models on disparate single-task datasets results in suboptimal performance due to poor ground truth.

Purpose of the Study:

• To introduce TaughtNet, a novel knowledge distillation framework for BioNER.
• To enable the training of a single multi-task BioNER model by effectively combining knowledge from multiple single-task models.
• To develop more efficient and deployable BioNER models suitable for resource-constrained environments.

Main Methods:

• TaughtNet employs a knowledge distillation approach, where a multi-task student model learns from both ground truth annotations and the outputs of specialized single-task teacher models.
• The framework integrates knowledge from diverse datasets, even those with single-entity annotations, to create a robust multi-task learning environment.
• Experiments were conducted on recognizing mentions of diseases, chemical compounds, and genes.

Main Results:

• TaughtNet demonstrated competitive performance against state-of-the-art baselines, achieving strong precision, recall, and F1 scores.
• The framework successfully trained smaller and lighter student models, enhancing deployability on hardware with limited memory and enabling faster inference.
• The approach shows potential for providing explainability in BioNER systems.

Conclusions:

• TaughtNet offers an effective solution to the challenge of limited annotated data in BioNER by leveraging knowledge distillation.
• The framework facilitates the creation of efficient, high-performing multi-task BioNER models suitable for practical, real-world applications.
• The study releases code and a pre-trained multi-task model to promote further research and development in the field.