Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Videos

Evaluating the Methodological Quality of Artificial Intelligence-Assisted Systematic Reviews: Protocol for a Mixed

Mohammad Jay^1,2, Mary Morgan³, Sharon Elizabeth Straus^2,4,5

¹Department of Medicine, Division of Endocrinology, University of Toronto, 2075 Bayview Avenue, Toronto, ON, M4N 3M5, Canada, 1 416-480-6705, 1 416-480-5761.

JMIR Research Protocols

|May 14, 2026

Summary

Related Concept Videos

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

"Straddling two worlds": complexities of spousal caregiving in amyotrophic lateral sclerosis during younger middle age.

BMC palliative care·2026

Same author

Exploring women's preferences for diabetes prevention programs after gestational diabetes: A qualitative descriptive study.

Diabetic medicine : a journal of the British Diabetic Association·2026

Same author

Moderate- versus high-activity radioactive iodine in intermediate-risk papillary thyroid cancer: a Canadian cohort study.

Journal of the Endocrine Society·2026

Same author

The Link Between Diabetes and Cancer: Converging Mechanistic and Epidemiologic Evidence.

Diabetes·2026

Same author

Effect of parathyroidectomy versus non-surgical management on renal outcomes in primary hyperparathyroidism: protocol for a systematic review and meta-analysis.

BMJ open·2026

Same author

FairCareNLP: An AI-driven patient review analyzer for healthcare.

PloS one·2026

This study compares AI-assisted and traditional systematic reviews (SRs), evaluating AI

Area of Science:

Meta-research and scientific methodology evaluation.
Application of artificial intelligence (AI) and large language models (LLMs) in scientific research.

Background:

Artificial intelligence (AI), including large language models (LLMs), is increasingly used in systematic review (SR) workflows.
Potential benefits of AI in SRs include accelerated searching, screening, data extraction, and reporting.
Uncertainty remains regarding AI's impact on SR methodological quality, reporting completeness, transparency, and reproducibility, compounded by inconsistent disclosure of AI use.

Purpose of the Study:

To compare the methodological quality of AI-assisted versus traditional SRs.
To refine, finalize, and apply an AI Transparency and Disclosure Index (AITDI).
To evaluate the reproducibility of AI-assisted SRs and explore knowledge user perspectives on AI in SRs.

Main Methods:

Keywords:

AMSTAR-2 PRISMA 2020 artificial intelligence evidence synthesis large language models meta-research reproducibility systematic review transparency

Related Experiment Videos

A 4-phase mixed methods meta-research study involving a matched cohort analysis of SRs published from 2023-2025.
Evaluation of methodological quality, reporting completeness, and risk of bias using established tools (AMSTAR 2, PRISMA 2020, ROBIS).
Application of a refined AITDI, assessment of reproducibility through comparative outputs, and analysis of qualitative interviews with knowledge users.

Main Results:

Study preregistration and search strategy finalization completed as of December 2025.
Title/abstract screening initiated, with data extraction planned for March-May 2026.
AITDI refinement, reproducibility testing, and qualitative interviews are scheduled through February 2027, with final analyses by April 2027.

Conclusions:

This study will offer one of the first empirical comparisons of AI-assisted versus traditional SRs in the LLM era.
Findings will guide responsible AI integration and inform the development of best practices for reporting and methodology.
Potential for developing AI-specific extensions for reporting guidelines (e.g., PRISMA-LLM, AMSTAR-LLM).