Scalable tracking of symptoms in the electronic health record using large language models in patients with central...

Advances in large language models (LLMs) provide a means for scalable tracking of patient symptoms in clinical trials and post-marking surveillance using the electronic health record (EHR). Therefore, we sought to validate symptoms extracted from the EHR using a LLM to scale symptom extraction from the EHR.

Across a dataset of 499 randomly chosen clinical notes from patients seen in a neuro-oncology clinic, GPT-4o annotated symptoms (headache, fatigue, nausea, anxiety, difficulties sleeping, numbness and tingling, rash, constipation, and diarrhea) with an average sensitivity and specificity of 0.97 relative to expert manual review. We then applied the LLM to an external dataset of 51,541 notes representing 1,642 patients to obtain real-world symptom prevalence for temozolomide, bevacizumab, lomustine, immune checkpoint inhibitors (ICI), and methotrexate.

In the external dataset, the average number of symptoms per note was 3.92, and the most common symptom was fatigue (83% of patients). Surprisingly, patients receiving ICIs suffered from the most symptoms (mean = 4.68) and those receiving methotrexate had the least (mean = 2.92). We also found that the prevalence of reported symptoms in this real-world cohort was often much greater than the prevalence of reported symptoms in clinical trials of similar treatment regimens.

Large language models offer the ability to scale symptom extraction from health records, which is crucial to understand symptom burden and power symptom-related interventions and studies in real-world patient cohorts.