A rule-based large language model screening framework for evidence synthesis in animal health: Feasibility evaluation...

Evidence synthesis is essential for summarizing existing knowledge and identifying research gaps in animal health, but study screening is resource-intensive and time-consuming. This study evaluated the feasibility of a rule-based large language model (LLM) screening framework developed to support protocol-aligned study selection in a veterinary scoping review on feed additives in calves. The framework was compared with consensus decisions from three independent human reviewers across two screening stages: title-and-abstract screening and full-text screening. Agreement between framework-generated and human consensus decisions was assessed descriptively, using human consensus as the operational reference standard. At title-and-abstract screening, overall agreement was 96.8% (211/218 records), with seven discordant decisions. At full-text screening, overall agreement was 97.5% (39/40 records), with one discordant exclusion. Discrepancies in stage 1 reflected ambiguity or incomplete reporting, as well as differences in eligibility criteria interpretation relative to the intended screening scope. In stage 2, discrepancies occurred when the framework diverged from human decisions because information was distributed across narrative and tabular elements that required integration. Operational screening time was substantially shorter for the LLM screening framework than for the human screening process. These findings support the feasibility of a structured, rule-based LLM screening framework as a decision-support tool for veterinary evidence synthesis when implemented with predefined eligibility criteria, documented prompts, and human oversight.