Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Video

Updated: Jun 28, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Evaluation of Large Language Models for Structured Data Extraction From Interstitial Lung Disease Clinical Notes:

Stephanie Ji Chen¹, Manoj Venkat Maddali^1,2,3, Curtis Langlotz⁴

¹Division of Pulmonary, Allergy, and Critical Care Medicine, Stanford Medicine, Stanford, CA, United States.

Journal of Medical Internet Research

|June 26, 2026

Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Macroscopic Pulmonary Fat Embolism Secondary to Intraosseous Line Placement: A Case Report.

Case reports in critical care·2026

Same author

Performance of a large language model on the reasoning tasks of a physician.

Science (New York, N.Y.)·2026

Same author

Successful GH Treatment of Hepatopulmonary Syndrome in Panhypopituitarism-related Advanced Liver Disease.

JCEM case reports·2026

Same author

A generalizable deep learning system for cardiac MRI.

Nature biomedical engineering·2026

Same author

Cultryx: Precision Diagnostic Stewardship for Blood Cultures Using Machine Learning.

medRxiv : the preprint server for health sciences·2026

Same author

A clinical environment simulator for dynamic AI evaluation.

Nature medicine·2026

This summary is machine-generated.

Large language models (LLMs) can accurately extract structured data from interstitial lung disease (ILD) clinical notes, matching human performance. These LLMs offer a faster and more cost-effective solution for clinical research and data management.

Area of Science:

Artificial Intelligence in Medicine
Natural Language Processing for Healthcare Data
Clinical Data Extraction and Analysis

Background:

Unstructured clinical notes present a significant bottleneck for structured data extraction, particularly in interstitial lung disease (ILD) research.
Manual interpretation of ILD notes for classification and registry creation is time-consuming and resource-intensive.
Large language models (LLMs) show promise in automating and streamlining this data extraction process.

Purpose of the Study:

To compare the efficacy of various LLMs in extracting structured binary data from unstructured ILD clinical notes.
To evaluate LLM performance in multiclass data extraction for ILD classification.
To assess the speed and cost-effectiveness of LLM-based data extraction compared to manual methods.

Keywords:

electronic health records information extraction interstitial lung diseases large language models natural language processing

Related Experiment Videos

Last Updated: Jun 28, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Main Methods:

Twelve LLMs were employed to extract binary answers to 10 ILD-specific clinical questions from 100 patient notes.
Two LLMs were further assessed for multiclass ILD classification data extraction.
Performance was benchmarked against consensus-derived ground truth from ILD physicians using accuracy, precision, recall, and F1-scores.

Main Results:

Seven LLMs achieved human-level accuracy (96.2%) in binary data extraction, processing notes in 1-2 seconds at minimal cost.
Models like gpt-oss-120b, o1, and o3-mini demonstrated the highest F1-scores.
Multiclass data extraction yielded lower accuracy (88.0%-91.1%) compared to binary extraction, with no significant difference between the two tested LLMs.

Conclusions:

LLMs demonstrate consistent, human-level accuracy in extracting structured binary data from ILD clinical notes, significantly improving speed and reducing costs.
While feasible, multiclass data extraction using LLMs is less accurate than binary extraction.
LLMs represent a powerful tool for enhancing the efficiency of clinical data extraction and supporting clinical research.