Grading Comparison of Standardized Patients and Pharmacist Faculty During a Mental Health Simulation

To analyze concordance between pharmacist evaluators and paid actor standardized patient (SP) evaluators when grading student pharmacists' performance in a simulated mental health patient interaction.

A mental health telehealth simulation was developed for second-year student pharmacists. Performance was concurrently assessed by an SP and a pharmacist evaluator. Differences in rubric-based assessments between the 2 evaluator types were subsequently analyzed as paired samples using the McNemar test.

All 59 enrolled students completed the simulation. There was strong agreement between SP and pharmacist evaluations for communication and professionalism metrics across the cohort. Discrepancies were observed on 8.6% of the rubric items and occurred most frequently on rubric items that required clinical judgment, such as "recommended appropriate over-the-counter options for the patient" (27% disagreement) or "no/limited use of medical jargon" (22% disagreement). Significant differences were seen, with SPs reporting that the student did not complete some tasks, whereas pharmacist evaluators reported the task as completed, such as "asked about duration of symptoms" and "recommended appropriate over-the-counter/nonpharmacologic options for the patient."

This study found strong concordance between SP and pharmacist evaluators for assessing observable communication behaviors and professionalism but identified significant differences in ratings for items requiring clinical judgment.