Contextual Word Embedding for Biomedical Knowledge Extraction: a Rapid Review and Case Study

Area of Science:

• Natural Language Processing (NLP)
• Biomedical Informatics
• Computational Linguistics

Background:

• Contextual word embedding models have advanced knowledge extraction from biomedical and healthcare texts.
• A comprehensive comparison of these models for biomedical applications is lacking.
• Existing research needs to evaluate model performance across diverse biomedical and clinical data.

Purpose of the Study:

• To conduct a scoping review of contextual word embedding models for biomedical knowledge extraction.
• To compare the performance of major models in text classification, named entity recognition, and question answering.
• To identify the most effective models for analyzing biomedical and clinical documents.

Main Methods:

• A scoping review identified 18 contextual word embedding models from 26 articles (2017-2021).
• A case study evaluated six representative models: ELMo, BERT, BioBERT, BlueBERT, Clinical BioBERT, and GPT-3.
• Performance was assessed using accuracy and F1 scores on datasets from tweets, NCBI, PubMed, and electronic health records.

Main Results:

• BioBERT demonstrated superior performance in analyzing general biomedical text.
• Clinical BioBERT achieved the best results when analyzing clinical notes.
• Model effectiveness varied depending on the specific NLP task and data type.

Conclusions:

• BioBERT and Clinical BioBERT are highly effective for biomedical and clinical text analysis, respectively.
• The findings provide valuable guidance for selecting appropriate word embedding models in NLP research.
• This study contributes to the efficient extraction of knowledge from complex health-related documents.