Transfer Learning with Clinical Concept Embeddings from Large Language Models

Knowledge exchange is crucial in healthcare, particularly when leveraging data from multiple clinical sites to address data scarcity, reduce costs, and enable timely interventions. Transfer learning can facilitate cross-site knowledge transfer, yet a significant challenge is the heterogeneity in clinical concepts across different sites. Recently, Large Language Models (LLMs) have shown significant potential in capturing the semantic meanings of clinical concepts and mitigating heterogeneity in biomedicine. This study analyzed electronic health records from two large healthcare systems to assess the impact of semantic embeddings from LLMs on local models, shared models, and transfer learning models. The results indicate that domain-specific LLMs, such as Med-BERT, consistently outperform in local and direct transfer scenarios, whereas generic models like OpenAI embeddings may need fine-tuning for optimal performance. This study emphasizes the importance of domain-specific embeddings and meticulous model tuning for effective knowledge transfer in healthcare. It remains essential to investigate the balance the balance between the complexity of downstream tasks, the size of training samples, and the extent of model tuning.