Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Data Validation01:03

Data Validation

5.2K
Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...
5.2K
Quality Assurance01:19

Quality Assurance

172
Quality assurance is the overarching term used to describe the activities employed to ensure the proper performance of a system. These activities can be classified into three categories: quality control, quality assessment, and internal corrective measures. Typically, these activities work cyclically: quality control is performed before and during the analysis, while quality assessment occurs during and after the investigation. Internal corrective measures are implemented based on the findings...
172
Methods of Documentation VI: Case Management Model01:15

Methods of Documentation VI: Case Management Model

612
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
612
Mechanistic Models: Compartment Models in Individual and Population Analysis01:23

Mechanistic Models: Compartment Models in Individual and Population Analysis

74
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
74
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches01:23

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

156
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
156
Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches01:14

Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches

197
Drug disposition in the body is a complex process and can be studied using two major approaches: the model and the model-independent approaches.
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
197

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models: Quantitative Study Using Large Language Models.

Journal of medical Internet research·2025
Same author

Zero- and few-shot prompting of generative large language models provides weak assessment of risk of bias in clinical trials.

Research synthesis methods·2024
Same author

The overview of the BioRED (Biomedical Relation Extraction Dataset) track at BioCreative VIII.

Database : the journal of biological databases and curation·2024
Same author

Integration of background knowledge for automatic detection of inconsistencies in gene ontology annotation.

Bioinformatics (Oxford, England)·2024
Same author

Integration of polygenic and gut metagenomic risk prediction for common diseases.

Nature aging·2024
Same author

Predicting Publication of Clinical Trials Using Structured and Unstructured Data: Model Development and Validation Study.

Journal of medical Internet research·2022
Same journal

Correction: Confidence Measurement Metrics in Multimodal Large Language Models for Ultrasound-Based Radiology Cases: Comparative Evaluation Study of Self-Reported, Consistency-Based, and Hybrid Methods.

Journal of medical Internet research·2026
Same journal

Centering Equity During Health Technology Innovation: Scoping Review of Methods and Research Adjustments to Promote Inclusive Coproduction.

Journal of medical Internet research·2026
Same journal

Comparative Effectiveness of AI-Assisted Telerehabilitation, Telerehabilitation, In-Person Care, and Usual Care for Chronic Nonspecific Low Back Pain: Bayesian Network Meta-Analysis.

Journal of medical Internet research·2026
Same journal

Effectiveness of WeChat Public Account Intervention Based on the Information-Motivation-Behavioral Skills Model Among College Students With Internet Addiction: Randomized Controlled Trial.

Journal of medical Internet research·2026
Same journal

Are Traditional Registries Becoming Obsolete in the Modern Digital Health Ecosystem?

Journal of medical Internet research·2026
Same journal

Detecting and Preventing Fraudulent Participation in Qualitative Research: Content Analysis of Two Multisite Studies.

Journal of medical Internet research·2026
See all related articles

Related Experiment Video

Updated: Aug 12, 2025

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

445

Automating Quality Assessment of Medical Evidence in Systematic Reviews: Model Development and Validation Study.

Simon Šuster1, Timothy Baldwin1,2, Jey Han Lau1

  • 1School of Computing and Information Systems, University of Melbourne, Melbourne, Australia.

Journal of Medical Internet Research
|February 1, 2023
PubMed
Summary
This summary is machine-generated.

This study introduces a machine learning system, EvidenceGRADEr, to automatically assess the quality of medical evidence in systematic reviews. It can accurately identify risk of bias and imprecision, aiding evidence synthesis.

Keywords:
automated quality assessmentbias detectioncritical appraisalevidence synthesissystematic reviews

More Related Videos

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
07:31

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Published on: May 15, 2020

7.2K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K

Related Experiment Videos

Last Updated: Aug 12, 2025

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

445
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
07:31

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Published on: May 15, 2020

7.2K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K

Area of Science:

  • Medical Informatics
  • Artificial Intelligence in Healthcare
  • Evidence-Based Medicine

Background:

  • Assessing medical evidence quality is crucial for systematic reviews.
  • Existing tools focus on individual studies, not entire evidence bodies.
  • A need exists for automated, comprehensive quality assessment tools.

Purpose of the Study:

  • To develop a machine learning system (EvidenceGRADEr) for overall quality rating of evidence bodies.
  • To provide finer-grained quality assessments based on Grading of Recommendation, Assessment, Development, and Evaluation (GRADE) criteria.
  • To construct a novel dataset for training and evaluating evidence quality assessment models.

Main Methods:

  • Algorithmically extracted quality data from Cochrane Database of Systematic Reviews summaries.
  • Defined evidence bodies by PICO criteria and assigned quality grades (high, moderate, low, very low).
  • Trained neural network variants using extracted data and compared predictions to author-assigned labels.

Main Results:

  • Created a dataset of 13,440 evidence bodies from 2252 systematic reviews (2002-2020).
  • EvidenceGRADEr achieved 0.78 F1 for risk of bias and 0.75 F1 for imprecision.
  • Performance was lower for inconsistency, indirectness, and publication bias (0.3-0.4 F1).
  • Binary classification of evidence quality (high/moderate vs. low/very low) reached 0.74 F1.

Conclusions:

  • Machine learning can automate the assessment of certain evidence quality dimensions (risk of bias, imprecision) in systematic reviews.
  • Challenges remain for automating assessment of indirectness, inconsistency, and publication bias.
  • This technology has the potential to significantly reduce reviewer workload and expedite evidence synthesis.