A comparison of word embeddings for the biomedical natural language processing

Area of Science:

• Biomedical Natural Language Processing (NLP)
• Machine Learning
• Computational Linguistics

Background:

• Word embeddings are crucial for capturing semantic relationships in biomedical NLP.
• Existing research often uses diverse corpora like Wikipedia or biomedical literature for training.
• Limited evaluation exists for word embeddings trained on different textual resources.

Purpose of the Study:

• To empirically evaluate and compare word embeddings trained from diverse corpora.
• To assess the performance of embeddings from clinical notes, biomedical publications, Wikipedia, and news.
• To understand the impact of training data on the semantic capabilities of word embeddings.

Main Methods:

• Trained word embeddings using electronic health record (EHR) data and PubMed Central (MedLit) articles.
• Utilized pre-trained GloVe and Google News embeddings for comparison.
• Conducted qualitative analysis (similarity inspection, visualization) and quantitative evaluation (intrinsic semantic similarity, extrinsic downstream tasks like information extraction).

Main Results:

• Embeddings from EHR and MedLit demonstrated superior ability in identifying semantically similar medical terms compared to GloVe and Google News.
• Intrinsic evaluation confirmed EHR embeddings closely align with human expert judgments on medical semantic similarity.
• Extrinsic evaluation showed EHR embeddings achieved the highest F1 score (0.900) for clinical information extraction, while Google News embeddings excelled in relation extraction (0.790).

Conclusions:

• Word embeddings trained on biomedical domain corpora (EHR, MedLit) better capture medical term semantics and align with expert judgment.
• No single word embedding model consistently outperforms others across all biomedical NLP tasks; however, their inclusion as features generally enhances performance.
• Biomedical domain-specific embeddings do not always outperform general domain embeddings in downstream biomedical NLP tasks.