Generating contextual embeddings for emergency department chief complaints.

Area of Science:

• Natural Language Processing
• Machine Learning in Healthcare
• Clinical Informatics

Background:

• Emergency department (ED) chief complaints are often unstructured free text.
• Deriving computational representations for these complaints is challenging but valuable.
• Bidirectional Encoder Representations from Transformers (BERT) is a powerful language model for contextual embeddings.

Purpose of the Study:

• To learn contextual embeddings for ED chief complaints using BERT.
• To derive compact and computationally useful representations for free-text chief complaints.
• To evaluate BERT's performance against baseline models like LSTM and ELMo.

Main Methods:

• Retrospective analysis of 2.1 million ED visits.
• Utilized BERT for classification task: predicting provider-assigned labels from free-text chief complaints.
• Compared BERT with Long Short-Term Memory (LSTM) and Embeddings from Language Models (ELMo).
• Performance measured by Top-k accuracy (k=1:5) on a held-out test set.
• Visualized embeddings using t-distributed stochastic neighbor embedding (t-SNE).

Main Results:

• BERT outperformed LSTM and ELMo in predicting chief complaint labels.
• Achieved high Top-5 accuracies (0.92-0.94) on datasets with fewer labels.
• t-SNE visualization revealed clinically meaningful clusters of related chief complaints.

Conclusions:

• BERT successfully learned rich representations of chief complaints despite noisy labels.
• Learned embeddings accurately predict provider-assigned labels and map semantically similar complaints.
• Potential applications include automatic mapping to structured fields and developing standardized chief complaint ontologies.