Comparison of two methods for ranking applicants for residency

Ranking radiology residency applicants is a complex process. Multiple factors, such as the variability in evaluation of candidates and the sometimes excessive subjectivity experienced, may influence the final outcome. To address inconsistencies, Yale University's selection committee integrated a mathematical model of ranking. The goal is to compare the mathematically generated rank list with the traditional committee-derived list to identify applicants with discrepancies between the two rank orders as a safety net to ensure that the final rank order list reflects true committee consensus. For three consecutive years, beginning with the 2006-2007 interview season, three rank order lists were compiled. The subjective list was developed by committee consensus on appropriate rank for each applicant. The mathematical list was developed using an equation to assign a score from each of an applicant's three interviewers, which were then averaged and arranged in descending order. These two lists were compared to identify applicants who had differences of 10 rank order positions. Identified applicants were reassessed and reassigned if necessary, forming the National Resident Matching Program (final) list submitted for the match. Over three years, 224 applicants were ranked in total, with 109 being reevaluated (49%) and 24 ultimately reassigned (11%). Discrepancies in rank on the two lists were identified and discussed. In some but not all cases, the discrepancies were remedied. Reasons for discrepancies are discussed. The mathematical method used in parallel with the subjective method has proved useful in identifying misplaced applicants and provided assurance that the final rank list reflects the committee's evaluation of each applicant.