A large language model for electronic health records.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Clinical Informatics

Background:

• Growing need for AI to interpret electronic health records (EHRs).
• Limited availability and size of existing clinical language models hinder medical AI development.
• Lack of understanding regarding the benefits of large-scale clinical language models.

Purpose of the Study:

• To develop and evaluate GatorTron, a novel, large-scale clinical language model.
• To investigate the impact of model parameter count and training data size on clinical NLP task performance.
• To assess the utility of large clinical language models for processing unstructured EHR data.

Main Methods:

• Developed GatorTron from scratch using over 90 billion words of text, including de-identified clinical data.
• Systematically evaluated GatorTron on five clinical Natural Language Processing (NLP) tasks: concept extraction, relation extraction, semantic textual similarity, natural language inference (NLI), and medical question answering (MQA).
• Examined the effects of scaling model parameters (up to 8.9 billion) and training data volume.

Main Results:

• GatorTron models demonstrated significant improvements across five clinical NLP tasks.
• Achieved notable accuracy gains, including 9.6% for NLI and 9.5% for MQA.
• Showcased the benefits of scaling both model size and training data for clinical language models.

Conclusions:

• Large clinical language models like GatorTron can substantially enhance the utilization of unstructured EHR data for medical AI.
• The developed GatorTron models offer a powerful tool for advancing healthcare delivery through improved AI applications.
• Publicly available GatorTron models facilitate further research and development in clinical NLP.