Related Experiment Video
Updated: Apr 13, 2026
Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
Evaluation framework for conversational AI agents in pharmacy education: A scoping review of key characteristics and
Sabrina Winona Pit1, Mohammad Hamiduzzaman1, Carl R Schneider2
1University Centre for Rural Health, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia.
Background:
Innovative and scalable training solutions, such as conversational artificial intelligence agents (CAIAs), are essential to meet the complex demands of pharmacy education and practice.
Objectives:
To explore and synthesize the key characteristics and outcome measures of CAIAs in pharmacy education and propose an evaluation framework for their use in pharmacy education.
Methods:
A scoping review identified studies published between 2020 and 2025. An evaluation framework was developed to capture key CAIA characteristics and outcome measures using the World Health Organization's digital health framework.
Results:
Of 961 studies screened, six met inclusion criteria. Five studies originated from English-speaking countries. CAIAs supported training in communication skills (n = 3), human resource management (n = 2), and HIV care (n = 1). Learner interaction with CAIAs was variable and often poorly described. Common features included scenario-based learning, immediate, real-time, automated feedback, interactive-learning, and modalities. Frequently evaluated outcomes were functionality (n = 5), user experience (n = 4), cost-benefit (n = 3), and user characteristics (n = 3). Educational outcome measures of confidence, knowledge and skills were included in three studies. All studies used text-based interaction; two included audiovisual elements, one combined text and voice, and two relied solely on text. Most (n = 5) involved single-user formats. Three studies were in evaluation stage one (feasibility/usability), two were in stage two (effectiveness) and one in stage three (efficacy). Outcomes demonstrated a low uptake for CAIAs, but indicate increased learner confidence, knowledge and communication skills. Eleven educational features were added to our evaluation framework and three educational outcome categories.
Conclusion:
This review contributed to the development of a framework to guide the design and evaluation of CAIAs in pharmacy education. CAIAs have been introduced in pharmacy education but remain in the early stages of adoption. Further research is needed to validate their effectiveness and expand their use in pharmacy as well test the use of the framework in other healthcare disciplines.
Related Concept Videos
01:16Non-equilibrium in the Cell
01:08Role of Communication in the Nursing Process III: Evaluation and Documentation
01:14SBAR II: Application of SBAR
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...
01:20Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
01:28Biopharmaceutics and Pharmacokinetics: Overview
01:26Drug Administration and Therapy Phases: Overview
The pharmaceutical phase focuses on leveraging the physicochemical properties of the drug to design and manufacture an effective product. Variants include orally administered tablets or capsules, topical creams or ointments, and parenteral-delivery solutions or emulsions.
The pharmacokinetic phase...