Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Fine-grained hierarchical multi-round iterative semantic optimization attack method for RAG systems.

Neural networks : the official journal of the International Neural Network Society·2026
Same author

Neutrophil Extracellular Traps in Heart Failure: From Pathophysiological Mechanisms to Therapeutic Targets.

Journal of inflammation research·2026
Same author

Effectiveness of static text adversarial methods on continuously updating models.

Neural networks : the official journal of the International Neural Network Society·2026
Same author

How Is Self-Compassion Associated with Prolonged Grief and Posttraumatic Stress After Bereavement? The Mediating Roles of Perceived Stigma and Anger.

Behavioral sciences (Basel, Switzerland)·2026
Same author

Downregulation of B7-H4 contributes to the synergistic effect of USP2a-targeted/anti-PD-1 combination therapy in EGFR mutant lung cancer.

Cancer immunology, immunotherapy : CII·2026
Same author

Ischemic stroke despite therapeutic anticoagulation in eosinophilic fasciitis with compartment syndrome and deep vein thrombosis: a case report.

Thrombosis journal·2026

Related Experiment Video

Updated: Jul 14, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Large language model-based multiagent collaboration for abstract screening toward automated systematic reviews.

Opeoluwa Akinseloyin1, Xiaorui Jiang2, Vasile Palade1

  • 1Centre for Computational Science and Mathematical Modelling, Coventry University, Puma Way, Coventry, CV1 2TT, United Kingdom.

Biology Methods & Protocols
|March 4, 2026
PubMed
Summary

Multiple large language models (LLMs) collaborating significantly improve systematic review abstract screening efficiency. Majority voting emerged as the top strategy, reducing workload by up to 68% while maintaining high recall.

Keywords:
abstract screeningensemblelarge language modelmultiagent systemsystematic review

More Related Videos

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
07:14

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models

Published on: December 23, 2025

Related Experiment Videos

Last Updated: Jul 14, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
07:14

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models

Published on: December 23, 2025

Area of Science:

  • Artificial Intelligence in Healthcare
  • Information Science
  • Biomedical Informatics

Background:

  • Systematic reviews (SRs) are crucial for evidence-based practice but are time-consuming, particularly abstract screening.
  • Current abstract screening methods are labor-intensive, hindering the efficient synthesis of research evidence.

Purpose of the Study:

  • To evaluate the effectiveness of multiple large language models (multi-LLMs) collaboration in enhancing the efficiency and reducing the cost of abstract screening for systematic reviews.
  • To compare different multi-LLM collaboration strategies against baseline question-answering (QA) approaches.

Main Methods:

  • Abstract screening was modeled as a QA task utilizing cost-effective LLMs.
  • Three multi-LLM collaboration strategies were tested: majority voting, multi-agent debate, and LLM-based adjudication.
  • Performance was assessed on 28 SRs from the CLEF eHealth 2019 benchmark using metrics like mean average precision (MAP) and work saved over sampling (WSS@95%).

Main Results:

  • Multi-LLM collaboration significantly outperformed individual QA baselines.
  • Majority voting achieved the highest MAP (0.462 and 0.341) and WSS@95% (0.606 and 0.680), indicating potential workload reduction up to 68% at 95% recall.
  • Multi-agent debate showed benefits for weaker models, while LLM-based adjudication was effective but more costly than voting or debate.

Conclusions:

  • Multi-LLM collaboration offers substantial improvements in abstract screening efficiency for systematic reviews, driven by model diversity.
  • Majority voting presents an optimal balance of high performance and low cost, making it a leading strategy.
  • Multi-agent debate remains a cost-effective option and a promising area for future research in technology-assisted review.