Testing bias: The hidden influence of race, gender, and identity in medical assessment

Implicit bias in medical education has been documented, but its influence on standardized assessments remains underexplored. This study examines whether patient demographic descriptors in clinical vignettes impact examinee multiple-choice question answer choice selection and clinical decision-making. A randomized, controlled survey was conducted among third- and fourth-year medical students and first-year residents. Participants (N = 376) answered clinical vignette multiple-choice questions in either a "standard group" (without demographic descriptors) or a "demographic group" (identical questions incorporating race, gender identity, or sexual orientation). Questions were grouped by question type (bias-dependent, bias-obscuring, and diagnostic-agnostic) based on whether the question stem enforced canonical pathologic associations with race and/or sociodemographic data (bias-dependent) or presented a noncanonical racial description (bias-obscuring) that could potentially lead medical trainees to select an incorrect answer choice. Performance was analyzed using Wilcoxon rank-sum tests (α < 0.0025). Participants in the standard group correctly answered more often bias-obscuring questions than those in the demographic group (66.5% vs. 58.8%, P < 0.001). Significant performance discrepancies were noted in questions related to homosexuality and sexually transmitted infection testing, sarcoidosis in Black females, Takayasu arteritis in individuals of Asian descent, and anemia in transgender individuals. Additionally, 52.4% of participants reported that race influenced their exam decision-making, while 64.1% and 55.9% indicated that gender identity and sexual orientation played roles. The majority (54.8%) attributed pro-White bias in medical decision-making to preclinical training. The inclusion of demographic descriptors, including race, sexual orientation, and gender identity, in standardized exam questions affects medical trainees' performance, potentially reinforcing bias.