Differences in language used to describe racial groups in emergency medicine standardized letter of evaluation

According to program directors in emergency medicine (EM), the standardized letter of evaluation (SLOE) is the most important component of the EM residency application. Understanding possible biases in SLOE language is critical for an equitable review process. Past studies have shown differences in the way medical students are described in narrative evaluations by race and gender; however, research on SLOE narratives has been limited to gender. This study seeks to evaluate narrative linguistic differences in applicant SLOEs by race.

This is a narrative analysis of all U.S. MD and DO SLOEs from applicants to the study institution in the 2022 application cycle. We used Linguistic Inquiry and Word Count (LIWC) to complete two analyses. Analysis 1 used frequency of words within 19 categories to evaluate differences between underrepresented minorities in medicine (URiM) and non-URiM applicants and within racial subgroups. Analysis 2 used LIWC to evaluate dichotomous use of 21 key words in these same groups. Linear mixed models were performed for each of the outcomes to evaluate for associations between URiM/non-URiM status or racial subgroup and each outcome.

Of the 809 unique applicants, 18.3% identified as URiM, 57.5% identified as White, 17.4% identified as Asian, 10% identified as Latinx, and 6.3% identified as Black. The analysis revealed applicants who are Black contained on average 0.537 (SE 0.154, Bonferroni-adjusted p = 0.010) percentage points more communal words when compared to White applicant SLOEs. URiM applicants had 0.322 percentage points more communal words (SE 0.102, Bonferroni-adjusted p = 0.030) compared to non-URiM SLOEs.

Applicants who were URiM or Black were more likely to be described with communal words than their peers. URiM applicants had twice the amount of empathic words in their SLOEs when compared to their non-URiM peers; however, this finding did not meet statistical significance (p = 0.053). Our study demonstrates that students who are URiM or Black are more likely to be described with language that, in STEM literature, has been associated with decreased hireability. This pattern may reflect the use of coded language in evaluations, which could hinder advancement of URiM residents and impact diversity in our field. These findings highlight the need for residency program directors and educators to critically examine evaluation language and implement strategies to ensure equitable assessment practices.