Related Experiment Video
Updated: Jan 17, 2026
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Testing the knowledge of artificial intelligence chatbots in pharmacology: examples of two groups of drugs
Marcin Mateusz Granat1, Aleksandra Paź1, Dagmara Mirowska-Guzel1
1Department of Clinical and Experimental Pharmacology, Medical University of Warsaw, Warsaw, Poland.
Objectives:
The study aimed to evaluate eight artificial intelligence chatbots (ChatGPT-3.5, Microsoft Copilot, Gemini, You.com, Perplexity, Character.ai, Claude 3.5, and ChatRTX) in answering questions related to two pharmacological topics taught during the basic pharmacology curriculum for medical students: antifungal drugs and hypolipidemic drugs.
Methods:
Chatbots' performance was assessed by answering 60 single-choice questions on antifungal and hypolipidemic drugs topics. The questions were designed to have four answers (a, b, c, and d), and the artificial intelligence (AI) role was to choose the proper one. The assessment was performed twice with a 1-year hiatus to determine if artificial intelligence chatbots' effectiveness changed over time. All the answers were checked for being right or wrong according to up-to-date pharmacology knowledge. To improve the clarity of results, to each score, a mark was assigned based on the grading system applied in our unit. Statistica software version 13.3 and Microsoft Excel 2010 were used for statistical analysis.
Results:
In 2023, the best results on the subject of antifungal drugs were obtained by Gemini (formerly Bard) and on the topic of hypolipidemic drugs by You.com (formerly YouChat). In 2024 Microsoft Copilot answered correctly the highest number of questions in both topics. The total results of all artificial intelligence chatbots in 2023 and 2024 were compared using t-test for dependent samples. Statistical analysis revealed that artificial intelligence chatbots improved over time in both pharmacological topics, but this change was not statistically significant (p = 0.784 for antifungal drugs subject and p = 0.056 for hypolipidemic drugs).
Conclusions:
The accuracy of AI chatbots' responses regarding antifungal and hypolipidemic drugs improved over one year, though not significantly. None of the tested AI systems provided correct answers to all questions within these pharmacological fields.
More Related Videos
07:02A Computerized Test Battery to Study Pharmacodynamic Effects on the Central Nervous System of Cholinergic Drugs in Early Phase Drug Development
Published on: February 11, 2019
13:18Network Pharmacology Prediction and Experimental Validation of Trichosanthes-Fritillaria thunbergii Action Mechanism Against Lung Adenocarcinoma
Published on: March 3, 2023
Related Concept Videos
01:25Drug Classes and Categories
01:26Drug Administration and Therapy Phases: Overview
The pharmaceutical phase focuses on leveraging the physicochemical properties of the drug to design and manufacture an effective product. Variants include orally administered tablets or capsules, topical creams or ointments, and parenteral-delivery solutions or emulsions.
The pharmacokinetic phase...
01:28Biopharmaceutics and Pharmacokinetics: Overview
01:28Structure-Activity Relationships and Drug Design
SAR studies the intricate relationship between a drug's chemical structure and biological activity. It focuses on understanding how modifications to a drug's structure can influence...
01:24Principles of Drug Action
Drugs can be agonists or antagonists. Like the endogenous ligands, agonists always bind and activate the target to produce a cellular response. Agonist binding induces a conformational change which in turn...
01:14Agonism and Antagonism: Quantification
To quantify these effects, researchers use a dose-response curve, which provides valuable information about the potency and efficacy of a drug. Potency refers to...