Exploring the Utility and Fairness of a Multi-institutional Situational Judgment Test

Area of Science:

• Pharmacy education
• Psychometrics
• Professionalism assessment

Background:

• Empathy, integrity, and teamwork are crucial for pharmacists.
• Situational judgment tests (SJTs) are used to assess these professional attributes.
• Understanding performance differences among student pharmacists is important for educational development.

Purpose of the Study:

• To pilot a newly developed SJT assessing empathy, integrity, and teamwork.
• To compare performance between first-year (P1) and fourth-year (P4) student pharmacists.
• To explore potential performance variations based on gender, race/ethnicity, and region.

Main Methods:

• Developed an SJT using best practices for assessing empathy, integrity, and teamwork.
• Piloted the SJT with P1 and P4 student pharmacists across five institutions.
• Adjusted scores for elevation and scatter to mitigate response style biases.

Main Results:

• Mean adjusted SJT scores were 86.3% for P1 and 87.2% for P4 students.
• Small institutional differences were noted, likely due to various factors.
• Higher scores were associated with older age, female gender, White race, and English as a first language.

Conclusions:

• SJT performance was comparable to existing literature, but subgroup differences emerged.
• Observed performance variations persisted after score adjustments.
• Further research is necessary to ascertain if these differences indicate test bias or other underlying factors.