Jove
Visualize
Contact Us

Related Concept Videos

Analysis of Population Pharmacokinetic Data01:12

Analysis of Population Pharmacokinetic Data

659
Analysis of population pharmacokinetic data involves studying the behavior of drugs within diverse populations to understand their pharmacokinetic parameters. Traditional pharmacokinetic methods typically involve collecting samples from a few individuals and estimating these parameters. While these methods are commonly used, they have limitations in capturing the variability in drug response among individuals or heterogeneous populations. Population pharmacokinetics is employed to address these...
659
Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs01:15

Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs

169
Body:Bioequivalence experimental study designs play a pivotal role in testing the effectiveness of various treatments. Key among these are the repeated measures, cross-over, carry-over, and Latin square designs. In the repeated measures design, each subject receives all treatments, allowing for temporal comparisons. This type of design is useful in reducing variability but requires careful planning to avoid bias.The cross-over design, an economical method, involves sequential administration of...
169
Study Designs in Epidemiology01:20

Study Designs in Epidemiology

868
Epidemiological study designs are fundamental tools for investigating the distribution, determinants, and control of health conditions in populations. They help researchers understand the relationships between exposures and outcomes, and they broadly fall into two categories: "observational" and "experimental" studies.
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...
868
Mechanistic Models: Compartment Models in Individual and Population Analysis01:23

Mechanistic Models: Compartment Models in Individual and Population Analysis

235
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
235
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches01:23

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

392
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
392
Bioavailability Study Design: Healthy Subjects Versus Patients01:15

Bioavailability Study Design: Healthy Subjects Versus Patients

137
Bioavailability studies are essential for evaluating a drug's therapeutic efficacy and understanding its absorption patterns under various physiological conditions. Conducting such studies on target patient populations provides more relevant data by simulating real-world disease states. However, practical challenges often necessitate the use of young, healthy adult volunteers as study subjects.Patients may exhibit altered drug absorption patterns due to the effects of the disease itself,...
137
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies
  1. Home
  3. Assessing The Feasibility And Acceptability Of A Bespoke Large Language Model Pipeline To Extract Data From Different Study Designs For Public Health Evidence Reviews.
  1. Home
  3. Assessing The Feasibility And Acceptability Of A Bespoke Large Language Model Pipeline To Extract Data From Different Study Designs For Public Health Evidence Reviews.

Related Experiment Video

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

1.3K

Assessing the Feasibility and Acceptability of a Bespoke Large Language Model Pipeline to Extract Data From Different

Zalaya Simmons1,2, Beti Evans1, Tamsyn Harris3

  • 1Research, Evidence and Knowledge Division, Chief Scientific Officer Group UK Health Security Agency (UKHSA) London UK.

Cochrane Evidence Synthesis and Methods
|November 6, 2025

View abstract on PubMed

Summary
This summary is machine-generated.
Keywords:
artificial intelligencedata extractionevidence synthesisfeasibilitylarge language modelpublic healthsystematic review

More Related Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.0K
Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System
05:10

Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System

Published on: December 11, 2016

10.2K

Related Experiment Videos

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

1.3K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.0K
Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System
05:10

Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System

Published on: December 11, 2016

10.2K

Large language models (LLMs) show promise for automating data extraction across diverse study designs, achieving 68% acceptability with human oversight. Further validation is needed before integration into review workflows.

Area of Science:

  • Artificial Intelligence
  • Medical Informatics
  • Data Science

Background:

  • Data extraction is a crucial, time-consuming component of evidence reviews.
  • Existing research highlights the efficiency gains of artificial intelligence (AI) and large language models (LLMs) for randomized controlled trials.
  • The efficacy of LLMs for data extraction across diverse study designs remains largely unexplored.

Purpose of the Study:

  • To assess the performance of a bespoke LLM pipeline (Retrieval-Augmented Generation with LLaMa 3-70B) for automating data extraction.
  • To evaluate the accuracy and reliability of LLM-driven data extraction across various study designs.
  • To determine the acceptability of LLM outputs for real-world evidence review applications.

Main Methods:

  • A Retrieval-Augmented Generation pipeline utilizing LLaMa 3-70B was developed for automated data extraction.
  • Accuracy was evaluated by comparing LLM extractions against human extractions for 173 data fields from 24 articles.
  • Reliability was assessed using the mean maximum agreement rate across 116 data fields from 16 studies.

    • Main Results:

    • 68% of the 173 evaluated data fields were rated as acceptable by human reviewers.
    • Acceptability varied by data field, with high scores for "objective," "setting," and "study design" (≥90%), but lower scores for "outcome" and "time period" (≤54%).
    • The mean maximum agreement rate for reliability was 0.71 (SD: 0.28), indicating variability across data fields.

    Conclusions:

    • LLMs, when combined with human quality assurance, can potentially support data extraction in evidence reviews encompassing multiple study designs.
    • Further performance enhancements and validation are necessary for the successful integration of LLM-based tools into systematic review workflows.