Extracting Medical Information From Unstructured Clinical Text Using Large Language Models to Enhance Health Care...

Area of Science:

• Artificial Intelligence in Healthcare
• Natural Language Processing
• Clinical Informatics

Background:

• Unstructured clinical text hinders data reuse and analysis in healthcare.
• Large language models (LLMs) show promise for structuring clinical data but require high-quality annotated training data.
• Scarcity of annotated data limits LLM application in healthcare.

Purpose of the Study:

• Develop and validate a scalable, privacy-preserving framework using synthetic data to fine-tune LLMs.
• Enable effective extraction of interoperable clinical information from unstructured text.
• Address limitations posed by scarce annotated training data.

Main Methods:

• Generated synthetic cancer discharge letters (n=75,000) from structured Fast Healthcare Interoperability Resources (FHIR) data using Qwen3-235B.
• Fine-tuned the MedGemma 27B medical language model on synthetic data paired with structured FHIR data.
• Evaluated model performance on synthetic and real-world discharge letters, comparing against general-purpose LLMs.

Main Results:

• Achieved high F1-scores for extracting diagnoses (0.84), tumor information (0.99), lab values (0.99), and medications (0.99).
• Demonstrated strong performance on real-world data with physicians, achieving 78.9% correctness for diagnoses and 93.0% for medications.
• Outperformed general-purpose LLMs in one-shot comparisons for most extraction tasks.

Conclusions:

• Synthetic text generation from structured clinical data is effective for training LLMs.
• This approach enables scalable extraction of interoperable, multi-entity clinical information.
• The framework overcomes data scarcity challenges for LLM application in healthcare.