Validation of Synthesa AI, a Large Language Model-Based Screening Tool for Systematic Reviews: Results From 9...

Systematic review screening underpins the evidence base for pharmacology and drug development but remains burdensome, error-prone, and resource-intensive. Synthesa AI, a large language model-based abstract screening tool, was developed to streamline this process by providing a transparent and prompt-driven framework for abstract screening. In this validation study, Synthesa AI was tested across 17 benchmark meta-analyses on 9 therapeutic domains relevant to pharmacology and clinical pharmacotherapy. The tool screened 270,626 abstracts retrieved from PubMed and Scopus. Synthesa AI successfully identified all 163 benchmark-included studies, achieving a sensitivity of 100% (95% confidence interval: 97.7%-100.0%) and a specificity of 99.4% (95% confidence interval: 99.37%-99.42%). Importantly, it reduced reviewer workload by 91.7%, with only 1797 abstracts requiring manual review. Beyond replication, the tool identified 32 additional relevant studies that had been missed in the original reviews, representing a 19.6% increase in evidence yield. These findings highlight the potential of Synthesa AI to enhance pharmacological evidence synthesis by improving the reproducibility and comprehensiveness of systematic reviews used to evaluate drug efficacy, safety, and therapeutic positioning. Synthesa AI represents a transformative solution for living systematic reviews and large-scale evidence integration, offering a rigorous and efficient alternative to traditional human-led screening in pharmacology research.