Language Differences by Race in the Narrative Section of the Emergency Medicine Standardized Letter of Evaluation

Discrimination and bias based on race/ethnicity permeate the medical education system. Racial disparities in assessment measures can ultimately impact applicants' Match results. Few studies to date have examined the narrative portion of the emergency medicine (EM) Standardized Letter of Evaluation (SLOE) for language differences by race. In this study we aimed to determine whether there were language differences by race in the narrative portion of the EM SLOE.

This study is an analysis of word category frequencies in the narrative portion of the SLOE for applicants applying to EM residency. The sample was drawn from the students who applied to the study institution in 2022. The narrative portion of the SLOE and other applicant factors were collected from the Electronic Residency Application Service (ERAS) applications and de-identified. We compared the number of SLOEs containing predefined keywords by race using chi2 analysis. Keywords were identified in six thematic word categories: agency; standout traits; ability; grindstone habits; achievement; and compassion. We performed logistic regression to determine whether any differences remained after controlling for other factors in the application.

Of 1,104 applicants to the institution, 2,288 SLOEs with self-identified race/ethnicity were available for analysis. Black and Hispanic applicants had higher proportions of SLOEs that contained a compassion word than White applicants (24.9% and 22.4% vs 16.9%, respectively). This finding persisted after controlling for other factors in the application for Black applicants (odds ratio 1.61, 95% CI 1.1-2.36]). There was no evidence of difference in word use by race across other thematic categories.

We found differences in the proportion of SLOEs containing compassion words in the narrative portion of the EM SLOE between Black and White applicants, with Black applicants being described with compassion language more frequently. However, we found no difference in any other word category, indicating less overall disparity than other narrative assessment studies.