Related Experiment Videos
Cochrane evaluation of (semi-)automated review methods: protocol for an adaptive platform study within reviews
Gerald Gartlehner1, Susan Banda2, Max Callaghan3
1Department for Evidence-Based Medicine and Evaluation, Cochrane Austria, University for Continuing Education, Krems, Austria; RTI International, Center for Health Economics, Methods & Evidence Synthesis, Research Triangle Park, NC, USA.
Journal of Clinical Epidemiology
|June 19, 2026
Summary
This study introduces an adaptive platform design to evaluate artificial intelligence (AI) tools for evidence synthesis. The flexible framework assesses AI effectiveness and efficiency in systematic reviews, promoting reliable integration of AI into research workflows.
Area of Science:
- Medical research methodology
- Health informatics
- Artificial intelligence in healthcare
Background:
- Systematic reviews are crucial for healthcare decisions but are time-consuming.
- Existing methods for evaluating artificial intelligence (AI) tools in evidence synthesis are underdeveloped.
- AI has the potential to enhance the efficiency and accuracy of evidence synthesis.
Purpose of the Study:
- To describe a novel study design for assessing AI tools in evidence synthesis workflows.
- To compare the effectiveness, efficiency, and usability of AI tools against traditional human-only methods.
- To establish a framework for the responsible integration of AI into systematic reviews.
Main Methods:
- An adaptive platform study-within-a-review (SWAR) design is employed, modeled after clinical platform trials.
- Multiple AI tools are concurrently evaluated against a human-only control across title/abstract screening, full-text screening, and data extraction.
- Performance metrics include accuracy, efficiency, response stability, error impact, and usability, adhering to Responsible use of AI in evidence SynthEsis (RAISE) principles.
Main Results:
- The study will yield comparative data on the performance and usability of AI tools in evidence synthesis.
- A flexible framework will be established for evaluating AI tools, addressing limitations of static assessments.
- The adaptive design allows for the dynamic inclusion or exclusion of AI tools based on interim analyses.
Conclusions:
- This protocol offers a novel approach to evaluating AI tools for evidence syntheses.
- Validating entire workflows, rather than individual technologies, will inform AI integration.
- The adaptive and flexible design can be adopted by other researchers, ensuring continued relevance as AI evolves.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:23Methods of Documentation V: CBE
Charting by Exception, or CBE, is a method of documentation used in healthcare, particularly in nursing, that focuses on documenting only significant or abnormal findings rather than recording every detail. This approach aims to streamline the documentation process, improve efficiency, and ensure that healthcare providers can quickly identify deviations from normalcy in patient assessments.
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...