Impact of Holistic Application Scoring on Interview Offers

Area of Science:

• Medical Education
• Surgical Residency Admissions
• Holistic Review

Background:

• Traditional surgical residency admissions often rely heavily on standardized test scores and academic metrics.
• This can inadvertently disadvantage applicants from diverse backgrounds or those who have overcome significant personal challenges.
• There is a need for more comprehensive screening tools that capture a wider range of applicant attributes.

Purpose of the Study:

• To develop and evaluate a holistic screening tool for surgical residency applications.
• To assess the tool's effectiveness in identifying successful candidates based on diverse skills and attributes.
• To incorporate an adversity score to recognize applicants' "distance traveled".

Main Methods:

• Developed a 4-domain application-scoring rubric based on ACGME Holistic Review criteria: academic potential, experiences, personal attributes, and clinical competency.
• Created an "adversity score" accounting for personal challenges such as homelessness, first-generation status, socioeconomic hardship, disability, or discrimination.
• Compared interview offer rates for top candidates ranked by adversity score, Step 2 score, academic potential, and overall holistic score.

Main Results:

• Applicants with the highest adversity scores (AS 4) had a 90% interview offer rate when using the holistic score, compared to 38% if only Step 2 scores were used.
• Using academic potential alone resulted in only 35% of AS 4 students being interviewed.
• The holistic overall score interviewed 80% of all students with any adversity score, whereas Step 2 scores alone interviewed 44% and academic potential alone interviewed 50%.

Conclusions:

• Holistic application review, including an adversity score, can mitigate implicit bias in surgical residency admissions.
• This approach better recognizes students who have overcome significant obstacles and possess the potential to be excellent residents.
• Considering a broader range of achievements and attributes leads to a more equitable and comprehensive selection process.