General Surgery Residents can Be a Reliable Resource in the Evaluation of Residency Applications

Surgical residents' ability to screen general surgery (GS) applicants has not been previously investigated. The objective of this study was to compare surgical residents' evaluation of Electronic Residency Application Service (ERAS) applicants to that of faculty using a standardized assessment instrument.

A prospective analysis of ERAS applications using a standardized assessment tool.

A university-affiliated, academic, county GS residency program.

Before the interview day, 51 ERAS (2013-2014) applications were reviewed by 10 different assessors (6 GS faculty, including the program director, and 4 GS residents), who evaluated applicants on 10 characteristics (subjective and objective) using a 5-point Likert scale, a total score, and a Global Rating Scale that ranked candidates into deciles.

There were a total of 510 assessments. In 8 of 10 individual domains the interrater reliability (IRR) between residents and faculty was good. The IRRs of the total score and global score were excellent. The Spearman ρ between the total score and final rank list were similar for faculty (-0.558) and residents (-0.592).

The excellent IRR score between the total and global scores of faculty and residents demonstrates the reliability of GS residents in evaluating ERAS applications. The low correlations between the total score and final rank are consistent with those in previous studies, in which the interview has been demonstrated to be the most important factor in determining final selection.