Does the ability to identify the construct on an empathy situational judgment test relate to performance? Exploring a...

The ability to identify criteria (ATIC) refers to an examinee's capacity to distinguish the construct being evaluated. Previous research indicates ATIC can be predictive of performance on some assessments. This exploratory study investigated the relationship between a participant's ability to identify criteria and their performance on an empathy situational judgment test (SJT), an assessment format used to measure social and behavioral attributes.

A 12-item empathy SJT was completed by 15 students and 15 pharmacists. During a cognitive interview, participants were asked what they believed each exam question measured. Responses were coded to determine whether participants stated "empathy" (indication of ATIC). The point-biserial correlation coefficient was calculated to explore the relationship of ATIC (correctly or not correctly identifying the item measured empathy) and performance on the SJT (total score).

Participants identified empathy 33.3% of the time, and it was the construct most often identified. Pharmacists (27.5%) identified empathy less often than students (39.2%). When empathy was identified as the construct, it was most often reported for items in a non-healthcare setting (56.3%) rather than a healthcare setting (43.7%) and for questions targeting affective empathy (71.3%) rather than cognitive empathy (28.7%). There were no statistically significant relationships with correctly identifying the construct and performance on individual items and the overall test.

There is inconclusive evidence that ATIC relates to performance on an empathy SJT. Additional research is needed to evaluate the role of ATIC and assessment performance to corroborate study results.