Distilling the knowledge from large-language model for health event prediction

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Computational Biology

Background:

• Electronic Health Records (EHRs) are crucial for health event prediction in Intensive Care Units (ICUs).
• Existing methods often focus on single data modalities (text or structured data), limiting prediction accuracy.
• Integrating multi-modal EHR data, including clinical text and structured data, remains a significant challenge.

Purpose of the Study:

• To propose the CKLE framework for health event prediction by leveraging large language models (LLMs) and multi-modal EHR data.
• To address the limitations of LLMs in handling non-textual data and the privacy concerns requiring local deployment.
• To improve the accuracy and interpretability of health event prediction models.

Main Methods:

• CKLE framework distills cross-modality knowledge from LLMs into a predictive model using multi-modal EHR data.
• Clinical text is refined and augmented with prompt learning, with embeddings generated by LLMs.
• A cross-modality knowledge distillation (KD) method is employed, featuring contrastive loss and patient similarity learning.

Main Results:

• CKLE achieved up to a 4.48% improvement in accuracy for heart failure and hypertension prediction compared to state-of-the-art models.
• The framework demonstrated superior performance on both normal and limited label settings.
• Feature importance analysis provided insights into salient features for cardiology disease prediction, aligning with medical knowledge.

Conclusions:

• CKLE effectively leverages LLM knowledge and multi-modal EHR data for enhanced health event prediction.
• The proposed cross-modality knowledge distillation method addresses LLM scalability and portability challenges in healthcare.
• CKLE offers a promising approach for real-world clinical applications, improving both prediction accuracy and interpretability.