Related Experiment Video
Updated: Jun 13, 2026
Comparing Bibliometric Analysis Using PubMed, Scopus, and Web of Science Databases
Published on: October 24, 2019
Analysis of article screening and data extraction performance by an AI systematic literature review platform
Kelsie Cassell1, Abiodun Ologunowa2, Majid Rastegar-Mojarad3
1Merck & Co., Inc., Rahway, NJ, United States.
Background:
Systematic literature reviews (SLRs) are critical to health research and decision-making but are often time- and labor-intensive. Artificial intelligence (AI) tools like large language models (LLMs) provide a promising way to automate these processes.
Methods:
We conducted a systematic literature review on the cost-effectiveness of adult pneumococcal vaccination and prospectively assessed the performance of our AI-assisted review platform, Intelligent Systematic Literature Review (ISLaR) 2.0, compared to expert researchers.
Results:
ISLaR demonstrated high accuracy (0.87 full-text screening; 0.86 data extraction), precision (0.88; 0.86), and sensitivity (0.91; 0.98) in article screening and data extraction tasks, but lower specificity (0.79; 0.42), especially when extracting data from tables. The platform reduced abstract and full-text screening time by over 90% compared to human reviewers.
Conclusion:
The platform has strong potential to reduce reviewer workload but requires further development.
