When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical...

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Machine Learning

Background:

• Large Language Models (LLMs) have advanced data analysis, but their use with tabular clinical data remains underexplored.
• Electronic Health Records (EHR) contain crucial numerical data for medical diagnostics and prognostics.

Purpose of the Study:

• To evaluate LLM embeddings derived from last hidden states for medical tasks using EHR data.
• To compare LLM embeddings against raw numerical EHR data as features for traditional machine learning models.

Main Methods:

• Utilized instruction-tuned LLMs in a zero-shot setting to represent abnormal physiological data.
• Employed eXtreme Gradient Boosting (XGBoost) as a traditional machine learning benchmark.
• Investigated prompt engineering techniques for zero-shot and few-shot LLM embeddings.

Main Results:

• Raw numerical EHR data generally outperformed LLM embeddings in medical machine learning tasks.
• Zero-shot LLM embeddings demonstrated competitive performance, indicating potential utility.
• Prompt engineering influenced the effectiveness of LLM embeddings.

Conclusions:

• LLM embeddings present a promising, though not yet superior, approach for feature extraction in medical ML.
• Further research is warranted to optimize LLM integration with EHR data for clinical applications.