Can Behavior-Based Interviews Reduce Bias in Fellowship Applicant Assessment?

Area of Science:

• Medical Education
• Psychometrics
• Diversity and Inclusion in Healthcare

Background:

• Trainee application components can introduce bias related to race or gender.
• Behavior-based interviews (BBIs) use structured questions to predict future behavior from past experiences.
• Assessing and mitigating bias in applicant selection is crucial for equitable training programs.

Purpose of the Study:

• To implement BBIs in a fellowship program.
• To compare applicant assessment by race and gender using standardized versus BBI-based tools.
• To evaluate the impact of BBIs on reducing assessment bias.

Main Methods:

• Developed BBIs and specific assessment tools in 2019-2020.
• Trained 6 of 15 faculty in BBIs.
• Applicants underwent 6 interviews (BBI or unstructured) and faculty completed standardized or BBI-specific assessments.
• Analyzed normalized scores and applicant quartiles by race and gender.

Main Results:

• Standardized assessments showed significant racial differences in scores (P=.02) and quartiles (P=.05).
• BBI scores and quartiles did not show significant racial differences (P=.18 and P=.17, respectively).
• No significant differences by gender were found for either assessment tool. Faculty and applicant surveys showed positive BBI feedback.

Conclusions:

• BBIs were successfully implemented and positively received by faculty and applicants.
• BBIs demonstrated a reduction in racial differences in applicant assessments.
• Structured tools like BBIs may improve applicant assessment by mitigating potential biases.