Related Experiment Video
Updated: Jun 26, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
SPELL: A scalable NLP method using regular expressions and large language models for clinical information extraction
Ricardo Kleinlein1, David W Bates2, Carolyn Guan1
1Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Background And Objectives:
Electronic health records (EHRs) contain valuable information for research and decision-making, but much resides in unstructured notes that are challenging to analyze at scale. We developed SPELL (Snippet-Primed rEgex LLM Pipeline), a scalable natural language processing workflow that combines regular-expression-based snippet retrieval with locally hosted large language model (LLM) inference to extract structured variables from large collections of clinical narratives.
Methods:
SPELL uses task-specific regular expressions to retrieve short context windows ("snippets") from unstructured texts and applies task-prompted LLM inference on snippets rather than full documents. All processing occurs within institutional computing environments. Accuracy was evaluated on randomly sampled, clinician-annotated benchmark sets of 50 documents per obstetric task, with separate retrieval-recall audits of 20 regex-negative documents per task. We evaluated accuracy and efficiency across three obstetric information-extraction tasks: numerical value (blood loss volume), date (estimated due date), and diagnosis (hemolysis, elevated liver enzymes, and low platelets [HELLP] syndrome). We quantified computational scalability using elapsed time, out-of-memory events, energy consumed, and GPU telemetry, and audited retrieval recall using clinician-annotated regex-negative notes enriched with relevant structured metadata. Generalizability was assessed on the public MT Samples corpus (5013 notes across 40 specialties) for ventricular tachycardia detection.
Results:
SPELL processed 31 million clinical notes spanning 1976-2024 from eight hospitals. Snippet-based inference reduced processing time by 71-87% versus full-document LLM inference and by >95% versus manual physician annotation. On the 50-document benchmark sets, snippet-based evaluation achieved 98% exact-match accuracy for blood-loss extraction, 92% exact-match accuracy for estimated-due-date extraction, and 94% accuracy with an F1-score of 0.97 for HELLP syndrome classification. As an exploratory external evaluation on MT Samples, ventricular tachycardia detection achieved 84% accuracy and an F1-score of 0.67.
Conclusions:
A hybrid regex-snippet-LLM pipeline can enable accurate and computationally efficient extraction from unstructured EHR narratives.
