Exploiting and assessing multi-source data for supervised biomedical named entity recognition.

Area of Science:

• Biomedical Natural Language Processing
• Bioinformatics
• Computational Biology

Background:

• Named entity recognition (NER) is crucial for extracting biomedical information from scientific text.
• Supervised machine learning models for NER heavily depend on annotated training data.
• Models trained and tested on the same data may overestimate performance and fail to generalize to diverse corpora.

Purpose of the Study:

• To evaluate the generalizability of biomolecular NER models across independent corpora.
• To investigate the impact of single-source versus multi-source training data on model performance.
• To identify factors contributing to model overtraining and performance discrepancies.

Main Methods:

• Aggregated published corpora for biomolecular entity recognition (genes, RNA, proteins, etc.).
• Employed a leave-corpus-out cross-validation strategy to assess model performance on independent datasets.
• Investigated orthographic features and annotation standard differences to explain performance variations.
• Conducted learning-curve-based power analysis to evaluate data quantity limitations.

Main Results:

• Model accuracies significantly decreased when tested on independent corpora, indicating poor generalizability.
• Combined use of multi-source training corpora resulted in more generalizable NER models.
• Achieved comparable performance with multi-source models as with single-source models, but with enhanced robustness.
• Found that model performance was often not limited by the quantity of annotated data.

Conclusions:

• Single-source training leads to overfitted models with limited generalizability in biomolecular NER.
• Aggregating diverse training corpora enhances model robustness and performance across different datasets.
• Future efforts should focus on creating diverse, multi-source training datasets for improved biomedical information extraction.