Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: May 24, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Evaluating Large Language Models for Extracting Clinical Recommendations from Practice Guidelines: A Preliminary

Rose Allington1, Nasim Mahmoodi1, Omid Pournik1

  • 1Department of Electronic, Electrical and Systems Engineering, School of Engineering, University of Birmingham, Birmingham.

Studies in Health Technology and Informatics
|May 23, 2026
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Quantifying the Viscoelastic Properties of Pancreatic Tissue: A Comparative Study of Human, Porcine and Engineered Hydrogel.

Annals of biomedical engineering·2026
Same author

A Hybrid Delphi-Inspired Expert-LLM Workflow for Efficient Evidence Screening in Systematic Reviews.

Studies in health technology and informatics·2026
Same author

Membership Inference or Data Split Bias? Identifying False Positives in Synthetic Medical Image Privacy Audits.

Studies in health technology and informatics·2026
Same author

Enhancing Ontology Engineering with Large Language Models: A Stage-Wise Human-in-the-Loop Study.

Studies in health technology and informatics·2026
Same author

OpenExtract: Automated Data Extraction for Systematic Reviews in Health.

Studies in health technology and informatics·2026
Same author

Secure and Enhanced Cyber-Threat Detection in IoMT Using Locally Deployed Large Language Models.

Studies in health technology and informatics·2026
Same journal

The Essential Components and Critical Conditions for Success in a Learning Health System in Oncology.

Studies in health technology and informatics·2026
Same journal

Use of Artificial Intelligence in Screening for Adolescent Idiopathic Scoliosis: A Scoping Review.

Studies in health technology and informatics·2026
Same journal

Movement Related Biomechanics in Adolescent Idiopathic Scoliosis: A Review of Reviews.

Studies in health technology and informatics·2026
Same journal

The Impact of Surgical Correction of Adolescent Idiopathic Scoliosis Using Posterior Spinal Fusion on Selected Radiological Parameters and Respiratory Function.

Studies in health technology and informatics·2026
Same journal

Acute Effect of Physio-logic® Exercises on Muscle Tone and Stiffness in Adolescent Idiopathic Scoliosis Patients: A Preliminary Study.

Studies in health technology and informatics·2026
Same journal

Effects of Integrated Music and Occupational Therapy on Motor and Autonomic Function in Children with Neurogenic Scoliosis.

Studies in health technology and informatics·2026
See all related articles

Large Language Models (LLMs) show promise for extracting clinical recommendations from Clinical Practice Guidelines (CPGs). DeepSeek and Grok models achieved over 90% accuracy in this knowledge extraction task.

Area of Science:

  • Medical Informatics
  • Artificial Intelligence in Healthcare
  • Clinical Knowledge Management

Background:

  • Clinical Practice Guidelines (CPGs) are essential for evidence-based healthcare.
  • Accessing and utilizing CPG content can be challenging for clinicians.
  • Large Language Models (LLMs) offer potential for automating information extraction from complex documents.

Purpose of the Study:

  • To evaluate the effectiveness of four different LLMs in extracting clinical recommendations from CPGs.
  • To assess the ability of LLMs to categorize extracted recommendations.
  • To compare LLM performance with and without an example set of extracted recommendations.

Main Methods:

  • Four distinct LLMs were tested for their ability to extract and categorize recommendations from CPGs.
Keywords:
AIClinical Practice GuidelinesKnowledge ExtractionLarge Language Models

Related Experiment Videos

Last Updated: May 24, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

  • Two testing conditions were employed: one with an example set and one without.
  • Accuracy and completeness of extracted recommendations were key evaluation metrics.

    • Main Results:

    • DeepSeek and Grok demonstrated superior performance among the tested LLMs.
    • These models achieved over 90% accuracy in extracting clinical recommendations.
    • The inclusion of an example set influenced the extraction and categorization process.

    Conclusions:

    • LLMs show significant potential for automating knowledge extraction from clinical guidelines.
    • Preliminary findings highlight both the capabilities and limitations of current LLMs in this domain.
    • Further research is needed to optimize LLM application for clinical knowledge management.