Related Experiment Video
Updated: Apr 16, 2026

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Evaluating automated or artificial intelligence search tools for evidence synthesis.
1robin.featherstone@cda-amc.ca, Acting Director, Research Information Services, Canada's Drug Agency, Ottawa, Ontario.
Canada's Drug Agency evaluated artificial intelligence (AI) search tools for evidence synthesis. Findings show AI tools have variable performance, recommending a "fit for purpose" approach for information retrieval tasks.
Area of Science:
- Information Science
- Health Informatics
- Artificial Intelligence in Research
Background:
- The Research Information Services team at Canada's Drug Agency sought to improve evidence synthesis processes.
- Evaluating automated and artificial intelligence (AI) search tools is crucial for advancing information retrieval science.
- A systematic approach was needed to assess the suitability of AI tools for specific research needs.
Purpose of the Study:
- To develop a replicable process for evaluating automated or AI search tools.
- To conduct a comparative analysis of leading AI search tools.
- To inform adoption decisions and enable tool comparison for evidence synthesis.
Main Methods:
- Inventoried 51 automated or AI search tools in Fall 2023.
- Developed a flexible evaluation instrument for tool assessment.
- Performed a comparative analysis of three top-ranked AI tools in Fall 2024.
Main Results:
- Automated or AI search tools demonstrated inconsistent and variable performance across information retrieval tasks.
- The evaluation highlighted limitations in current AI tool capabilities for complex research needs.
- No single AI tool was universally superior for all information retrieval functions.
Conclusions:
- Information Specialists at Canada's Drug Agency should adopt a "fit for purpose" strategy for AI tool utilization.
- AI search tools are best leveraged for specific tasks or project types rather than general application.
- Further research is needed to refine AI tool evaluation and integration into evidence synthesis workflows.
Related Concept Videos
Non-equilibrium in the Cell
Deductive Reasoning
For example, a researcher can deduce specific predictions...
The Evidence for Evolution
Systematic Sampling Method
Systematic sampling is one of the simplest methods...
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Heuristics
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...