Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Data Validation01:03

Data Validation

Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Impact of COVID-19 pandemic on surgical cancer care disparities according to socioeconomic status.

Public health in practice (Oxford, England)·2026
Same author

Prospective evaluation of genomics-guided off-label treatment.

Nature·2026
Same author

Halogenated persistent organic pollutants in selected processed small fish species from Ghana.

Food chemistry·2025
Same author

Induction therapy with BRAF/MEK inhibitors versus upfront ipilimumab/nivolumab in poor prognostic advanced melanoma.

European journal of cancer (Oxford, England : 1990)·2025
Same author

Perforator mapping for head and neck reconstructive surgery: a novel personalized approach using magnetic resonance angiography based 3D models and 3D-printing.

Frontiers in oncology·2025
Same author

Checkpoint Blockade Combinations in Tumor Mutational Burden/Load-High Tumors: Insights from the Atezolizumab + Bevacizumab and Nivolumab + Ipilimumab Cohorts in the Drug Rediscovery Protocol.

Clinical cancer research : an official journal of the American Association for Cancer Research·2025
Same journal

Retrospective analysis of real-world clinical use of comprehensive genomic profiling in solid tumors in Finland 2017-2020.

ESMO real world data and digital oncology·2026
Same journal

Development and evaluation of a large language model-based, retrieval-augmented generation application for query response in early oncology clinical trials.

ESMO real world data and digital oncology·2026
Same journal

The evolving physician-AI relationship: a five-tier framework for integrating intelligent systems into clinical practice and medical education.

ESMO real world data and digital oncology·2026
Same journal

Real-world patient characteristics and utilization patterns in patients with cholangiocarcinoma who received pemigatinib in the United States of America.

ESMO real world data and digital oncology·2026
Same journal

Real-world obesity prevalence and history in 79 271 patients receiving systemic anticancer therapy across 13 cancer types in England (2013-2023).

ESMO real world data and digital oncology·2026
Same journal

Accelerating real-world data collection using large language models in rare neoplasms: a bone sarcoma example.

ESMO real world data and digital oncology·2026
See all related articles
  1. Home
  3. Validating Large Language Model-assisted Data Extraction From Clinical Notes.
  1. Home
  3. Validating Large Language Model-assisted Data Extraction From Clinical Notes.

Related Experiment Video

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Validating large language model-assisted data extraction from clinical notes.

J W van Koevorden1,2, N Aben3, V Struben3

  • 1Department of Head and Neck Surgery, Antoni van Leeuwenhoek-Netherlands Cancer Institute, Amsterdam, the Netherlands.

ESMO Real World Data and Digital Oncology
|May 25, 2026

View abstract on PubMed

Summary
This summary is machine-generated.

Large language models (LLMs) significantly reduce clinical documentation time in head and neck oncology, achieving high accuracy. Human oversight is crucial for AI-assisted documentation tools to support clinical workflows effectively.

Keywords:
administrative burdeninformation extractionlarge language model (LLM)oncologyreal-world datavalidation

More Related Videos

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
07:50

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts

Published on: September 20, 2018

Related Experiment Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
07:50

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts

Published on: September 20, 2018

Area of Science:

  • Artificial Intelligence in Medicine
  • Clinical Informatics
  • Natural Language Processing

Background:

  • Healthcare professionals face significant documentation burdens impacting efficiency and patient safety.
  • Large language models (LLMs) offer a potential scalable solution for automating data extraction from unstructured clinical notes.
  • This study assesses LLM accuracy and clinical impact for structured data extraction in head and neck oncology.

Purpose of the Study:

  • To compare the accuracy and efficiency of LLM-driven data extraction against manual physician extraction.
  • To evaluate the clinical impact and error types associated with LLM data extraction.
  • To determine the potential of LLMs in streamlining clinical documentation for oncology consultations.

Main Methods:

  • A prospective validation study analyzed 1482 pages of clinical documentation from 60 patients.
  • A pretrained open-source LLM and two physicians extracted data across 29 categories.
  • Six clinical experts evaluated 2555 extracted values for accuracy, precision, recall, and F1 scores, categorizing errors.

    • Main Results:

    • LLM extraction accuracy ranged from 74% (pathology) to 90% (patient characteristics).
    • Manual extraction exhibited 29% interobserver disagreement, while LLM hallucinations were rare (0.16%) and low impact.
    • LLM extraction reduced average case time from 8.6 to 1.9 minutes (P < 0.001).

    Conclusions:

    • LLMs can effectively reduce documentation time in clinical settings while maintaining acceptable accuracy.
    • Human oversight is essential when implementing AI-assisted documentation tools.
    • Findings support further research into AI tools for clinical practice to enhance documentation efficiency.