Using large language models to enhance clinically-driven missing data recovery algorithms in electronic health...

Area of Science:

• Health Informatics
• Clinical Data Management
• Artificial Intelligence in Healthcare

Background:

• Electronic health record (EHR) data frequently contain missing values and errors, impacting data quality and clinical research.
• Traditional chart reviews are effective but resource-intensive, limiting their application to large patient cohorts.
• Previous work introduced a roadmap protocol using auxiliary diagnoses to impute missing EHR data.

Purpose of the Study:

• To evaluate the accuracy and scalability of a roadmap-driven algorithm for recovering missing EHR data.
• To compare the performance of LLM-enhanced roadmaps against traditional chart reviews.
• To assess the feasibility of applying these algorithms to large-scale EHR datasets.

Main Methods:

• Developed and refined roadmap algorithms using International Classification of Diseases, 10th revision (ICD-10) codes.
• Iteratively enhanced roadmaps with large language models (LLMs) and clinical expertise to identify auxiliary diagnoses.
• Validated algorithm performance against expert chart reviews for 100 patients and tested scalability on 1000 patients from an extensive EHR.

Main Results:

• Expert chart reviews recovered 12% (49/413) of missing EHR values in 100 patients.
• LLM-enhanced roadmap algorithms recovered 20%-22% (83-89/413) of missing values.
• A clinician-approved LLM-enhanced algorithm recovered 18% (73/413) of missing values, balancing expansion and clinical relevance.
• Application to 1000 patients increased the median non-missing EHR values per patient from 6 to 7.

Conclusions:

• Clinically-driven algorithms, augmented by LLMs, can accurately recover missing EHR data, comparable to manual chart reviews.
• These algorithms demonstrate scalability for application to large EHR datasets.
• Future work could extend these methods to address other data quality issues in EHRs.