How Do We Choose? A Review of Residency Application Scoring Systems

As the USMLE Step 1 Board exam moves to a pass/fail system there will be fewer objective measurements available to evaluate students applying to residency programs. Thus, there is a need for a reliable, validated method of screening applicants based on all areas of their application. To this end, we conducted a literature review search to examine previously described residency application screening tools.

A PubMed search was conducted using the keywords "residency," "applicant," "scoring," "algorithm," and "ranking." The search was limited to the last 10 years, and only articles written in English with the full-text available were included. The initial search yielded 512 results. Titles and abstracts were evaluated for inclusion and 11 articles met criteria for full-text evaluation. An additional 6 articles were excluded with reason following the full-text evaluation.

A total of 5 papers were included in our descriptive analysis. Villwock et al. used the open-source STAR algorithm to create an initial interview list based on program-specific desirable attributes. Bowe et al. attempted to develop a screening tool based on the 6 ACGME competencies that could accurately predict a resident's performance. Similarly, Lyons et al. worked with an outside consulting firm to develop a screening tool that used several situational judgment questions to assess desired competencies for first year residents. Schenker et al. developed an evaluation process that used a combination of a standardized screening tool and semistructured interviews to produce a final rank list. Hu et al. created a screening tool for pharmacology residency applicants based on specific domains.

Several residency application scoring systems have been evaluated for use in the initial screening process, but there is no consensus on which system is superior and whether or not those systems are successful in selecting the "best" candidates.