Active learning: a step towards automating medical concept extraction.

Area of Science:

• Natural Language Processing
• Machine Learning
• Medical Informatics

Background:

• Manual annotation of clinical text for medical concept extraction is labor-intensive.
• Developing accurate models for medical concept extraction requires substantial annotated data.

Purpose of the Study:

• To present an automatic, active learning-based system for medical concept extraction from clinical free-text reports.
• To quantify the reduction in annotation effort achieved by active learning.
• To evaluate the robustness of an incremental active learning framework.

Main Methods:

• Compared active learning (AL) with a fully supervised approach using Conditional Random Fields (CRFs).
• Investigated two AL selection criteria: least confidence and information density.
• Evaluated the impact of incremental learning versus standard learning on model robustness.
• Utilized two clinical datasets: i2b2/VA 2010 and ShARe/CLEF 2013.

Main Results:

• Active learning achieved comparable effectiveness to supervised learning with significantly reduced annotation.
• Annotation effort savings reached up to 77% for sequences, 57% for tokens, and 46% for concepts.
• Active learning savings were at least double that of random sampling.

Conclusions:

• Incremental active learning offers a robust and effective method for medical concept extraction.
• This approach substantially reduces the manual annotation burden in clinical natural language processing.