Transfer Learning with Clinical Concept Embeddings from Large Language Models.

Area of Science:

• Biomedical Informatics
• Artificial Intelligence in Healthcare

Background:

• Knowledge exchange in healthcare is vital for data scarcity and timely interventions.
• Clinical concept heterogeneity across sites poses a challenge for cross-site knowledge transfer.
• Large Language Models (LLMs) show promise in understanding clinical semantics and reducing heterogeneity.

Purpose of the Study:

• To evaluate the impact of LLM-generated semantic embeddings on healthcare data.
• To compare the performance of local, shared, and transfer learning models using LLM embeddings.
• To assess the effectiveness of domain-specific versus generic LLMs in cross-site knowledge transfer.

Main Methods:

• Analysis of electronic health records from two large healthcare systems.
• Utilizing semantic embeddings derived from domain-specific (Med-BERT) and generic (OpenAI) LLMs.
• Evaluating model performance in local, shared, and transfer learning settings.

Main Results:

• Domain-specific LLMs (Med-BERT) demonstrated superior performance in local and direct transfer learning scenarios.
• Generic LLMs (OpenAI embeddings) may require fine-tuning for optimal results in cross-site applications.
• LLM embeddings significantly impacted the performance of various modeling approaches.

Conclusions:

• Domain-specific LLM embeddings are crucial for effective healthcare knowledge transfer.
• Meticulous model tuning is essential for leveraging LLMs in heterogeneous clinical data.
• Future research should explore the interplay between task complexity, data size, and model tuning for LLM applications in healthcare.