Active learning reduces annotation time for clinical concept extraction.

Area of Science:

• Natural Language Processing
• Machine Learning in Healthcare
• Clinical Informatics

Background:

• Manual annotation of clinical free text is time-consuming.
• Efficiently extracting concepts from clinical data is crucial for research and healthcare applications.
• Active learning strategies can potentially optimize the annotation process.

Purpose of the Study:

• To quantify annotation time savings using active learning query strategies versus supervised learning and random sampling.
• To evaluate the efficiency gains from active learning-assisted pre-annotations compared to de novo annotation.

Main Methods:

• User study involving manual annotation of discharge summary reports.
• Comparison of annotation time for 'from scratch' annotation with active learning versus random sampling.
• Evaluation of reviewing time for active learning pre-annotations versus de novo annotation.

Main Results:

• Active learning reduced annotation time by up to 35% (vs. supervised) and 28% (vs. random sampling) when annotating from scratch.
• Reviewing active learning pre-annotations offered an additional 20% reduction in annotation time compared to de novo annotation.
• A strong correlation exists between the number of concepts annotated and the time required for active learning approaches.

Conclusions:

• Active learning plays a vital role in decreasing the time needed for manual annotation of clinical concepts.
• Both 'from scratch' annotation and reviewing pre-annotations benefit from active learning strategies.
• The findings highlight active learning's potential to streamline clinical data annotation workflows.