Screening Routine Clinical Notes for Epilepsy Surgery Candidates Using Large Language Models

Epilepsy surgery is severely underutilized despite proven efficacy, with substantial under-referral of eligible patients in routine clinical practice. This study evaluated the potential role of large language models (LLMs) as decision-support tools for screening unstructured clinical notes to identify epilepsy surgery candidates and stratify them according to prognostic indicators.

We retrospectively analyzed free-text medical records in a non-English language (Hebrew) from 110 patients in a tertiary epilepsy clinic. Six LLMs (Gemini 2.5 Pro, 2.5 Flash, 2.0 Flash; GPT-5, GPT-5 mini; and o4-mini) were prompted to extract surgical eligibility criteria, parameters of the Seizure Freedom Scale (SFS) for surgical prognostication, completion of presurgical evaluations, and previous surgical consideration. Model outputs were compared with expert manual review.

Model performance in identifying core eligibility parameters demonstrated high sensitivity (up to 1.00) and specificity (up to 0.96), with favorable predictive values (PPV up to 0.92, NPV up to 1.00). Majority voting yielded near-perfect sensitivity (1.00 in this cohort) for identifying surgical eligibility. Notably, 45% (13/29) of patients meeting surgical criteria had no prior consideration of surgery. Models demonstrated high accuracy in SFS score evaluation (sensitivity 0.95, specificity 0.93) and strong performance in identifying completed presurgical evaluations.

These findings suggest the potential role of LLMs to act as decision-support tools for identifying patients who may benefit from surgical evaluation but have not been recognized in routine care. This is supported by the models' high performance in correctly identifying eligible patients, as well as prognostic parameters. As this performance was achieved using off-the-shelf general-purpose models applied directly to raw, non-English clinical notes, it suggests a practical and scalable screening approach across diverse clinical settings.