Transformers and large language models are efficient feature extractors for electronic health record studies

Area of Science:

• Medical Informatics
• Computational Linguistics
• Clinical Research

Background:

• Electronic health records contain abundant free-text data.
• Challenges in information extraction often lead to the use of less specific clinical codes.

Purpose of the Study:

• To evaluate the efficacy of NLP and LLMs for extracting infection types from free-text clinical notes.
• To compare the performance of advanced models against traditional methods and existing coding systems.

Main Methods:

• Utilized a dataset of 938,150 hospital antibiotic prescriptions from Oxfordshire, UK.
• Trained various models, including Bio+Clinical BERT, GPT-3.5, and GPT-4, to infer infection types from free-text indications.
• Compared model performance using F1 scores on internal and external test datasets.

Main Results:

• A fine-tuned Bio+Clinical BERT model achieved the highest performance (average F1 score 0.97-0.98).
• Zero-shot GPT-4 matched traditional NLP models without labeled data (F1 0.71-0.86).
• Free-text indications identified specific infection sources 31% more often than ICD-10 codes.

Conclusions:

• Transformer-based models show significant potential for widespread use in medicine.
• These models can enhance information extraction from structured free-text records.
• Improved data extraction can facilitate better clinical research and patient care.