Can Large Language Models Reduce the Cost of Extracting Data from Electronic Health Records for Research?

Area of Science:

• Biomedical Informatics
• Computational Linguistics

Background:

• Unstructured electronic health records (EHR) contain valuable medical data.
• Extracting information from EHRs is typically done manually or via natural language processing (NLP).
• Current NLP methods are only cost-effective for large datasets (~6500 records).

Purpose of the Study:

• To investigate if large language models (LLMs) can reduce NLP costs for smaller EHR projects.
• To assess the accuracy and feasibility of an LLM-based NLP framework.

Main Methods:

• Developed an NLP pipeline using an open-source LLM to extract breast cancer biomarker data (ER, PR, HER2) from EHRs.
• Compared the LLM pipeline's development time and performance against a rule-based (RB) NLP pipeline.
• Made the LLM pipeline code publicly available.

Main Results:

• The LLM pipeline achieved performance competitive with manual data extraction.
• Hands-on development time for the LLM pipeline was approximately 38% of that for the RB pipeline.

Conclusions:

• LLMs offer lower hands-on development costs than traditional NLP techniques.
• LLMs require significant computational resources, which can be costly.
• LLMs show potential for making NLP economically viable for smaller EHR projects if computational expenses are managed.