A study of deep active learning methods to reduce labelling efforts in biomedical relation extraction.

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Machine Learning

Background:

• High-quality labelled data is crucial for developing predictive methods in biomedical research.
• Building large, annotated datasets for biomedical relation extraction (bioRE) is limited by time and expertise (annotation bottleneck).

Purpose of the Study:

• To investigate the effectiveness of Active Learning (AL) strategies in overcoming annotation limitations for bioRE tasks.
• To benchmark different AL strategies and evaluate their impact on model performance and annotation efficiency.

Main Methods:

• Benchmarking six AL strategies on seven bioRE datasets using PubMedBERT as the base model.
• Evaluating strategies based on Area Under the Learning Curve (AULC) and intermediate results (F1-score, accuracy, precision, recall).

Main Results:

• Uncertainty-based AL strategies (Least-Confident, Margin Sampling) showed superior performance in F1-score, accuracy, and precision.
• The Core-set strategy excelled in recall, outperforming other methods.
• AL strategies reduced annotation needs by 6% to 38% to achieve performance comparable to full dataset training.

Conclusions:

• Active Learning is vital for efficient construction of high-quality bioRE datasets.
• AL methods significantly reduce the labeling effort required for optimal deep learning model performance.
• Uncertainty and diversity sampling strategies offer distinct advantages for different performance metrics in bioRE.