Comparing neural language models for medical concept representation and patient trajectory prediction

Area of Science:

• Computational linguistics
• Bioinformatics
• Health informatics

Background:

• Effective representation of medical concepts is vital for analyzing electronic health records (EHRs).
• Neural language models offer potential for deriving medical concept representations from clinical data.
• Comparative performance and semantic encoding of different language models in this domain require further investigation.

Purpose of the Study:

• To evaluate word2vec, fastText, and GloVe for creating medical concept embeddings.
• To assess the semantic meaning captured by these empirical representations.
• To compare their effectiveness in downstream tasks like outcome and trajectory prediction.

Main Methods:

• Trained word2vec, fastText, and GloVe on patient trajectories from a large EHR dataset.
• Assessed semantic encoding by comparing empirical embeddings with biomedical terminologies.
• Evaluated performance in predicting patient outcomes (length-of-stay, readmission, mortality) and future medical codes.

Main Results:

• fastText embeddings showed highest similarity (0.88-0.92) to theoretical clusters for diagnosis, procedure, and medication codes.
• word2vec and GloVe outperformed fastText in outcome prediction (AUROC up to 0.85).
• GloVe and fastText showed strong performance in predicting medical codes within patient trajectories.

Conclusions:

• Subword information is crucial for learning medical concept representations.
• Global embedding vectors are better suited for high-level tasks like trajectory prediction.
• These models can effectively encode clinical meaning from EHR data, highlighting the potential of machine learning for semantic data encoding.