Toward digital twins in the intensive care unit: a medication management case study

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Natural Language Processing in Healthcare

Background:

• Digital twins offer potential for personalized medicine by simulating patient responses.
• Large language models (LLMs) show promise in processing clinical notes for decision support.
• Current LLM applications in healthcare require domain-specific adaptation for optimal performance.

Purpose of the Study:

• To evaluate the efficacy of LLM-based digital twins for treatment recommendations in intensive care units (ICUs).
• To determine if specialty-specific fine-tuning improves accuracy compared to general or zero-shot models.
• To assess the impact of Low-Rank Adapters (LoRA) fine-tuning on LLM performance.

Main Methods:

• LLaMA-3 LLM was fine-tuned using Low-Rank Adapters (LoRA) on ICU physician notes (discharge summaries).
• Medications were masked to create training and testing datasets from the Medical Information Mart for Intensive Care III (MIMIC-III) dataset.
• Performance was evaluated using BERTScore and ROUGE-L on a medical ICU dataset, compared against zero-shot baselines.

Main Results:

• Models fine-tuned on medical ICU notes achieved the highest BERTScore (0.842).
• Specialty-specific fine-tuning outperformed models trained on other ICU specialties or mixed datasets.
• Zero-shot baseline models demonstrated the lowest performance, underscoring the necessity of model training.

Conclusions:

• Specialty-specific training significantly enhances treatment recommendation accuracy in LLM-based digital twins.
• Context-specific fine-tuning is crucial for developing effective digital twins for clinical decision support.
• These findings provide foundational insights for advancing personalized clinical decision support systems.