Enhancing clinical concept extraction with contextual embeddings.

Area of Science:

• Natural Language Processing (NLP)
• Clinical Informatics
• Machine Learning

Background:

• Neural network embeddings have advanced NLP and clinical concept extraction.
• Recent models like ELMo and BERT offer improved performance but lack standardized integration practices for clinical tasks.

Purpose of the Study:

• To explore and compare advanced embedding methods (ELMo, BERT) against traditional word embeddings (word2vec, GloVe, fastText) for clinical concept extraction.
• To establish best practices for integrating novel embedding representations into clinical NLP workflows.

Main Methods:

• Evaluated off-the-shelf and MIMIC-III pretrained clinical embeddings.
• Compared traditional word embeddings with contextual embeddings (ELMo, BERT) on four concept extraction corpora (i2b2 2010, i2b2 2012, SemEval 2014, SemEval 2015).
• Analyzed the impact of pretraining duration and explored semantic information in contextual embeddings.

Main Results:

• Contextual embeddings pretrained on clinical data achieved new state-of-the-art performance across all concept extraction tasks.
• The top model surpassed existing methods with F1-scores of 90.25, 93.18 (partial), 80.74, and 81.65.
• Demonstrated superior performance of contextual embeddings over traditional methods.

Conclusions:

• Contextual embeddings hold significant potential for advancing clinical concept extraction.
• These advanced embeddings encode valuable semantic information beyond traditional word representations.
• The study provides a foundation for integrating cutting-edge NLP models into clinical applications.