Gender and Inconsistent Evaluations: A Mixed-methods Analysis of Feedback for Emergency Medicine Residents

Area of Science:

• Medical Education
• Gender Studies in Medicine
• Emergency Medicine Training

Background:

• Prior research indicates similar initial evaluations for male and female emergency medicine (EM) residents.
• A significant gender gap emerges by the final year, with women receiving lower scores than men.
• Understanding the reasons behind this evaluation gap is crucial for addressing gender inequality in medical training.

Purpose of the Study:

• To investigate discrepancies between numerical scores and the sentiment of textual comments in EM resident evaluations.
• To analyze gender-based differences in the consistency of evaluation feedback over time.
• To identify potential sources of gender bias in the assessment of EM residents.

Main Methods:

• A multicenter, longitudinal, retrospective cohort study analyzed 11,845 evaluations from four US academic EM training programs (2013-2015).
• Quantitative and qualitative methods were used to assess 202 residents (135 men, 67 women) evaluated by 151 attending physicians.
• A real-time, mobile-based, direct-observation tool captured numerical scores and textual comments.

Main Results:

• Numerical scores correlated more strongly with positive comment sentiment for men (r=0.38) than women (r=-0.26).
• Women were 11% more likely to receive positive comments with lower scores and negative/mixed comments with higher scores.
• Men showed expected improvement in evaluations over time (PGY-1 to PGY-3), while women's evaluations were less consistent.

Conclusions:

• Women EM residents experience more inconsistent evaluations than men, both in score-comment alignment and career progression feedback.
• Findings suggest gender inequality in how attending physicians evaluate residents.
• Training for attendings is recommended to provide clear, consistent, and equitable feedback to all residents, irrespective of gender.