Related Experiment Video
Updated: Aug 6, 2026
09:52
Setting Up a Stroke Team Algorithm and Conducting Simulation-based Training in the Emergency Department - A Practical Guide
Published on: January 15, 2017
End-to-End Pipeline Integrating Local Small Language Models and Machine Learning for Data Extraction and Stroke
Junsu Kim1, Ji Hoon Kim1,2, Arom Choi1,2
1Department of Emergency Medicine, Yonsei University College of Medicine, Seoul 03722, Republic of Korea.
Computational and Structural Biotechnology Journal
|July 24, 2026
Summary
This study introduces a privacy-preserving pipeline using a small language model (SLM) to extract valuable information from unstructured clinical text for stroke outcome prediction. The validated SLM pipeline significantly improves data accuracy, enabling reliable risk stratification and decision support.
Area of Science:
- Computational linguistics and natural language processing in healthcare.
- Clinical informatics and data science for evidence generation.
- Machine learning applications in medical research and patient care.
Background:
- A significant portion of clinical data exists as unstructured text, hindering its use for research and decision-making.
- Small language models (SLMs) offer potential for information extraction but raise concerns regarding hallucinations and data privacy.
- There is a critical need for locally deployable SLMs to ensure secure and reliable processing of sensitive healthcare data.
Purpose of the Study:
- To develop and validate an end-to-end pipeline for extracting information from unstructured clinical text.
- To integrate this pipeline with stroke outcome prediction models for improved clinical decision support.
- To demonstrate a privacy-preserving and efficient method for transforming clinical text into structured, usable data.
Main Methods:
- A Llama 3 8B small language model (SLM) was fine-tuned using low-rank adaptation and 4-bit quantization for local deployment.
- A multi-tiered validation strategy, including rule-based checks, retrieval-augmented generation, similarity flagging, and human-in-the-loop review, was implemented.
- Structured data from 767 ischemic stroke patients were used to train predictive models for 3-month outcomes, with TabPFN utilized for risk stratification.
Main Results:
- Initial text extraction accuracy of 64.9% improved to 86.0% after automated validation and 97.0% after human review.
- High F1 scores were achieved for template-based (F1 > 0.90) and narrative extraction (F1 = 0.87), with accurate NIHSS score extraction (MAE = 0.853).
- The TabPFN model demonstrated reliable risk stratification for poor stroke outcomes with an AUROC of 0.816.
Conclusions:
- The developed pipeline offers a privacy-preserving and efficient solution for processing unstructured clinical text.
- Combining SLMs with multi-tiered validation and predictive modeling transforms raw text into valuable structured data.
- This approach serves as a proof-of-concept for broader deployment in stroke outcome research and clinical decision support systems.