Integrating large language models for enhanced predictive analytics in healthcare.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Health Informatics

Background:

• Physicians require reliable tools for time-sensitive clinical outcome forecasting.
• Current predictive models face adoption barriers due to data handling, development, and workflow integration complexities.

Purpose of the Study:

• To introduce a novel framework (Hopkins LLM) for developing and deploying clinical large language models (LLMs).
• To leverage structured electronic health records (EHRs) data for multi-task-capable predictive engines.
• To minimize implementation barriers for clinical decision-support tasks.

Main Methods:

• Utilized the LLaMA architecture (7 billion parameters) for pre-training and fine-tuning.
• Trained and tested on a dataset of 42,160 patients from Johns Hopkins Health System.
• Validated across three external health systems and four prediction tasks (readmissions, mortality, ICU admissions, treatment recommendations) on 1,329 patients.

Main Results:

• Hopkins LLM achieved a mean area under the receiver operating characteristic curve (ROC-AUC) of 0.84 [0.82, 0.88].
• Demonstrated a significant 0.28 improvement over zero-shot baseline LLMs (p<0.05).
• Successfully predicted clinical and operational outcomes across diverse patient cohorts and health systems.

Conclusions:

• LLMs show promise as unified, user-friendly clinical prediction systems.
• The Hopkins LLM framework enhances point-of-care decision-making by reasoning across diverse data.
• This approach offers a scalable solution for integrating advanced predictive analytics into clinical workflows.