A study of active learning methods for named entity recognition in clinical text.

Area of Science:

• Clinical Natural Language Processing (NLP)
• Machine Learning (ML)
• Bioinformatics

Background:

• Named Entity Recognition (NER) is crucial for clinical NLP, but ML models require extensive expert-annotated data.
• Annotation is costly and time-consuming, necessitating efficient methods to reduce this burden.

Purpose of the Study:

• To develop and evaluate active learning (AL) methods for clinical NER.
• To compare the effectiveness of uncertainty-based, diversity-based, and baseline sampling strategies against passive learning (random sampling).

Main Methods:

• Simulated AL experiments were conducted on the 2010 i2b2/VA NLP challenge corpus (349 clinical documents).
• Existing and novel AL algorithms were categorized into uncertainty-based, diversity-based, and baseline strategies.
• Performance was evaluated using learning curves (F-measure vs. annotation cost) and Area Under the Learning Curve (ALC) scores.

Main Results:

• Uncertainty sampling algorithms demonstrated superior performance in ALC compared to all other methods.
• The best uncertainty sampling method achieved an F-measure of 0.80 with 66% fewer sentence annotations and 42% fewer word annotations than random sampling.
• Diversity-based methods showed moderate improvements over random sampling, with one method saving only 7% annotation effort.

Conclusions:

• Active learning, especially uncertainty-based approaches, can significantly reduce annotation costs for clinical NER tasks.
• Further real-time evaluation is recommended to confirm the practical benefits of AL in clinical NER.