Related Experiment Video
Updated: Apr 16, 2026

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Evaluating automated or artificial intelligence search tools for evidence synthesis
1robin.featherstone@cda-amc.ca, Acting Director, Research Information Services, Canada's Drug Agency, Ottawa, Ontario.
Abstract:
To advance information retrieval science for producing evidence syntheses at Canada's Drug Agency, the Research Information Services team developed a replicable process to evaluate automated or artificial intelligence (AI) search tools. The team inventoried 51 tools in the fall of 2023 and built a flexible evaluation instrument to inform adoption decisions and enable comparison between tools. Building on this foundational evaluation work, the team further conducted a comparative analysis on three top-ranked tools in the fall of 2024. The investigation confirmed that these automated or AI tools have inconsistent and variable performance for the range of information retrieval tasks performed by Information Specialists at Canada's Drug Agency. Implementation recommendations from this study informed a "fit for purpose" approach where Information Specialists leverage automated or AI search tools for specific tasks or project types.
Related Concept Videos
Non-equilibrium in the Cell
Deductive Reasoning
For example, a researcher can deduce specific predictions...
The Evidence for Evolution
Systematic Sampling Method
Systematic sampling is one of the simplest methods...
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Heuristics
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...