Enhancing EHR-based pancreatic cancer prediction with LLM-derived embeddings

Area of Science:

• Oncology
• Medical Informatics
• Artificial Intelligence

Background:

• Pancreatic cancer (PC) diagnosis is often delayed due to a lack of early symptoms and effective screening.
• Genetic or familial factors account for only about 10% of PC cases.
• Longitudinal electronic health record (EHR) data presents a potential avenue for early PC detection.

Purpose of the Study:

• To develop and evaluate a predictive model for early pancreatic cancer detection using EHR data.
• To enhance the model's performance by incorporating large language model (LLM)-derived embeddings of medical conditions.

Main Methods:

• A predictive model was developed utilizing LLM-derived embeddings of medical condition names from EHR data.
• The model was validated across two major medical centers: Columbia University Medical Center and Cedars-Sinai Medical Center.
• Performance was assessed using Area Under the Receiver Operating Characteristic curves (AUROCs) for 6-12 month predictions.

Main Results:

• LLM embeddings improved 6-12 month PC prediction AUROCs from 0.60 to 0.67 at one site and 0.82 to 0.86 at another.
• Excluding data from 0-3 months prior to diagnosis further boosted AUROCs to 0.82 and 0.89.
• The model demonstrated a significantly higher positive predictive value (0.141) compared to traditional risk factors (0.004).

Conclusions:

• EHR-based predictive models incorporating LLM embeddings show promise for early pancreatic cancer detection.
• This approach can identify high-risk individuals, including those without known risk factors or genetic predispositions.
• The model may serve as an independent tool for identifying individuals at high risk for pancreatic cancer.