Health system-scale language models are all-purpose prediction engines

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing for Healthcare
• Clinical Decision Support Systems

Background:

• Physicians face daily time-constrained decisions.
• Clinical predictive models aid decision-making but are complex to implement.
• Existing models struggle with data processing and deployment.

Purpose of the Study:

• To develop and evaluate a novel clinical language model for predictive tasks.
• To demonstrate the utility of unstructured clinical notes for training AI models.
• To create an all-purpose clinical predictive engine with low-resistance development.

Main Methods:

• Leveraged natural language processing advances to train a large language model (NYUTron) on medical language.
• Fine-tuned NYUTron across diverse clinical and operational predictive tasks.
• Evaluated model performance on readmission, mortality, length of stay, and insurance denial predictions.

Main Results:

• NYUTron achieved an Area Under the Curve (AUC) of 78.7-94.9% across tasks.
• Demonstrated AUC improvements of 5.36-14.7% compared to traditional models.
• Showcased benefits of pretraining with clinical text and site-specific fine-tuning.

Conclusions:

• Clinical language models can serve as effective, general-purpose predictive engines.
• Unstructured clinical notes are a valuable data source for AI in healthcare.
• NYUTron shows potential for real-time clinical guidance at the point of care.