Inequities in the National Clinical Assessment Tool for Medical Students in the Emergency Department

The National Clinical Assessment Tool for Emergency Medicine (NCAT-EM) was designed to standardize medical student assessments during emergency medicine clinical rotations. While multiple assessment tools implemented in medical education have been prone to inequities, it remains unknown how student and rater demographics impact NCAT-EM scores. In this study we examined how a student's gender and status as under-represented in medicine (URM) affected NCAT-EM scores.

This was a retrospective cohort study of all NCAT-EM assessments of clerkship medical students at a single institution in 2022. We performed mixed-effect ordinal logistic regression analyses to determine the association between the seven NCAT-EM domains (history/physical, prioritized differential, formulation of plans, observation/monitoring, emergency management, communication, and global assessment) and student gender, as well as the NCAT-EM domains and students' URM status (specifically in domains of race and ethnicity). We adjusted our analyses for the site of rotation, time, the rater's role (attending or resident), and rater demographics (gender, URM status). We then evaluated the interaction in gender concordance and URM-status concordance on outcomes.

A total of 1,881 NCAT-EM assessment forms were submitted on 142 students completed by 266 raters. There were no significant associations between student gender and NCAT-EM ratings across the seven domains. We found an association between URM students and lower scores in multiple NCAT-EM domains, including global assessment (odds ratio [OR] 0.50, CI 0.25-0.99, P = .01); history/physical (OR 0.38, CI 0.19-0.77, P = .01); and prioritized differential (OR 0.47, CI 0.26-0.88, P = .02). This effect was moderated by a significant positive interaction effect with URM concordance between raters and students in the prioritized differential and observation/monitoring domains.

This is the first study to highlight differences in both gender and status as under-represented in medicine within the nationally implemented NCAT-EM assessment tool. Women students were overall rated similarly across the NCAT-EM domains compared to men, with no association of gender on ratings. However, students' URM status was associated with lower scores in multiple NCAT-EM domains. This finding was mitigated by URM concordance between faculty and resident raters. Our findings support the need for additional studies to understand bias and inequities in the application of the NCAT-EM tool nationally.