Related Experiment Video
Updated: Aug 6, 2026
05:47
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Efficacy of a Large Language Model Data Extraction System in Evidence Reviews for Emerging Infectious Diseases: A
Masahiro Ishikane1, Yuki Kataoka2,3,4,5,6,7, Yasushi Tsujimoto4,8,9,10
1Disease Control and Prevention Center, National Centre for Global Health and Medicine, Japan Institute for Health Security, Shinjuku, Tokyo, Japan.
Open Forum Infectious Diseases
|July 24, 2026
Summary
Large language models (LLMs) can accelerate evidence synthesis by assisting in data extraction from publications. This study found LLM assistance reduced extraction time by approximately 23% with no loss in accuracy.
Area of Science:
- Medical Informatics
- Public Health
- Artificial Intelligence in Medicine
Background:
- Rapid evidence synthesis is crucial for emerging infectious disease outbreaks, but traditional methods are often too slow.
- Large language models (LLMs) show potential to expedite evidence synthesis through automated data extraction from scientific literature.
Purpose of the Study:
- To compare the efficiency and accuracy of LLM-assisted data extraction versus manual extraction for scientific publications.
- To evaluate the potential of LLMs in accelerating evidence synthesis during public health emergencies.
Main Methods:
- A randomized crossover trial involving five experienced reviewers comparing LLM-assisted (OpenAI's o3 model) and manual data extraction from mpox-related articles.
- Primary outcome: task completion time. Secondary outcomes: extraction accuracy and adverse events.
- Statistical analysis using mixed-effects models to compare the two extraction conditions.
Main Results:
- LLM-assisted extraction averaged 7.9 minutes faster per article (27.5 min vs. 34.5 min), a reduction of approximately 23%.
- Extraction accuracy was 100% in both LLM-assisted and manual conditions.
- No adverse events were reported during the study.
Conclusions:
- LLM assistance shows promise in reducing data extraction time for evidence synthesis without compromising accuracy.
- While further development and validation are needed, LLM integration could significantly benefit rapid evidence synthesis during public health crises.
