Impact of Holistic Application Scoring on Interview Offers

We aimed to develop a holistic screening tool for surgical residency application processing to capture the diverse skills and attributes of the applicant, based on characteristics most commonly associated with success in our residency program.

We developed an application-scoring rubric with 4 domains based on ACGME Holistic reviewing criteria: academic potential, experiences, personal attributes, and clinical competency. We scored academic potential based on a composite of Step 2 score, MSPE tier rank, surgery clerkship grade, college honors, publications, and presentations. An additional score accounted for personal adversity overcome or "distance travelled". This included previous homelessness, first-generation college student or immigrant status, noted socioeconomic hardship, disability overcome, or other stated personal experience of discrimination including underrepresented in medicine status. We sorted the list of top 200 candidates by adversity score, Step 2 score, academic potential score, and total overall score to compare the groups in terms of interview offers.

We are an academic surgical residency program housed within a private medical college in the Northeast region.

All categorical applicants to our program were managed with our holistic screening approach.

There were 29 students with the highest adversity score (AS) of 4 and 26 (90%) were selected to interview based on holistic overall score and reviewer comments. Fourteen students had an AS of 3, and 12 (86%) were selected to interview. Twenty-five students had an AS of 2, and 23 (92%) were selected to interview. If Step 2 score alone had been used to determine which applicants should be interviewed, only 11 students (38%) with an AS of 4 would have been interviewed. If the academic potential alone was used for screening, only 10 (35%) of students with an AS of 4 would have been interviewed. Taking all students with any adversity score into account (n = 70), when screened with only Step 2 scores, just 31(44%) would have been interviewed. When ranked by academic potential score, 35 (50%) would have been interviewed. When applying our holistic overall score alone, 56 (80%) would have been interviewed.

Performing a holistic application review and ranking students not only by standardized exam scores, but also considering other history of academic achievement, personal attributes, experiences in leadership or service, and clinical competency can allow for mitigation of implicit bias. Allowance for an adversity score can help programs recognize students who have significantly longer "distance traveled" to make it through medical education and who have the potential to be fantastic residents.