Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches01:23

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

503
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
503
Data Validation01:03

Data Validation

7.1K
Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...
7.1K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Upadacitinib enhances platelet procoagulant responses in a subset of healthy individuals and patients with SLE: a pilot study.

Lupus science & medicine·2026
Same author

Unbiased distance correlation with sample-size-aware confidence bounds for comparative omics network analysis.

Frontiers in bioinformatics·2026
Same author

Examining emergency department utilisation in people with Crystal Arthropathies: a 10-year analysis of visits in a publicly funded universal healthcare system.

EULAR rheumatology open·2026
Same author

Global advances in health artificial intelligence: a workforce imperative.

Lancet (London, England)·2026
Same author

Prognostic data extraction harnessing a privacy-preserving large language model: a clinician-AI collaborative retrospective evaluation in head and neck oncology.

NPJ precision oncology·2026
Same author

A reliable and verified dataset of functional miRNA-mRNA target interactions for gene regulation studies.

Scientific data·2026

Related Experiment Video

Updated: Feb 28, 2026

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
07:31

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Published on: May 15, 2020

8.2K

The Alberta Risk of Bias Assessment Tool (AQAT:RoB) for the Evaluation of Medical Large Language Model

Carrie Ye1,2,3, Joseph Ross Mitchell1,4, Daniel C Baumgart1

  • 1University of Alberta, 8-130 Clinical Sciences Building11350 83 Ave NW, Edmonton, CA.

Journal of Medical Internet Research
|February 25, 2026
PubMed
Summary
This summary is machine-generated.

A new tool, the Alberta Risk of Bias Assessment Tool for LLM-QA studies (AQAT:RoB), has been developed to evaluate the validity and risk of bias in Large Language Model Question-Answer studies, enhancing AI safety in healthcare.

More Related Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.7K

Related Experiment Videos

Last Updated: Feb 28, 2026

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
07:31

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Published on: May 15, 2020

8.2K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.7K

Area of Science:

  • Medical Informatics
  • Artificial Intelligence in Healthcare
  • Research Methodology

Background:

  • Large Language Models (LLMs) offer transformative potential in healthcare but require rigorous evaluation.
  • Existing AI reporting guidelines and risk-of-bias tools are insufficient for LLM Question-Answer (LLM-QA) studies.
  • A critical gap exists in assessing the safety and effectiveness of LLM-QA studies due to a lack of specialized assessment tools.

Purpose of the Study:

  • To develop the Alberta Risk of Bias Assessment Tool for LLM-QA studies (AQAT:RoB).
  • To systematically evaluate the validity and risk of bias in LLM-QA studies.
  • To address the need for a specialized tool for assessing LLM-QA study quality.

Main Methods:

  • Conducted two literature reviews on LLM-QA quality assessment tools and studies.
  • Developed a draft AQAT:RoB informed by literature reviews.
  • Refined the tool through a modified-Delphi process, consensus meeting, and validation with 4 evaluators on 16 studies.

Main Results:

  • The AQAT:RoB comprises five high-level domains and nine sub-domains, rating bias as low, high, or unclear.
  • Pilot validation demonstrated high inter-rater reliability with 86.1% agreement and a Cohen's Kappa of 0.70.
  • The tool includes 'Support for Judgement' and 'Type of Bias' for each sub-domain.

Conclusions:

  • The AQAT:RoB shows promising initial reliability for assessing LLM-QA studies.
  • The tool requires further refinement, external validation, and periodic updates.
  • AQAT:RoB is a crucial step towards ensuring the safety and effectiveness of LLMs in healthcare.