Hawks and Doves: Adjusting for Bias in Residency Interview Scoring

Area of Science:

• Medical Education
• Residency Admissions
• Surgical Training

Background:

• Individual interviews are crucial for residency selection.
• Interviewer scoring variability can introduce bias.
• This bias may impact candidate rankings in the match process.

Purpose of the Study:

• To determine if interviewers or interview timing introduce bias in residency interview scores.
• To develop a method for adjusting interview scores to mitigate bias.

Main Methods:

• Analysis of 1,305 interview scores from 91 interviewers over 3 years at a general surgery residency program.
• Linear mixed models used to assess bias by year and interview order.
• Calculation of an individualized adjustment score for each interviewer based on average deviation.

Main Results:

• No significant difference in scores based on interview year or order.
• Significant interviewer bias was identified, with a progressive bias evident between low and high scoring interviewers (p < 0.0001).
• Application of the calculated adjustment score eliminated the bias, with no significant difference between interviewer ranking and scores (p = 0.32).

Conclusions:

• Residency interview scores exhibit significant interviewer bias.
• This bias can substantially influence an applicant's final rank.
• An adjustment score effectively reduces interviewer bias in interview scores, aiding fairer candidate evaluation.