General Surgery Residency Application Evaluation in a USMLE Step 1 Pass/Fail World: A Retrospective Comparison

Area of Science:

• Medical Education
• Residency Admissions
• Surgical Training

Background:

• The United States Medical Licensure Exam (USMLE) Step 1 historically served as a critical metric for residency program selection.
• A recent shift to a pass/fail system for USMLE Step 1 has altered traditional applicant evaluation processes.
• Residency programs now face the challenge of assessing candidates without this standardized score.

Purpose of the Study:

• To analyze the impact of USMLE Step 1 scores on applicant ranking within a General Surgery residency program.
• To determine how the removal of Step 1 scores affects the distribution and stratification of applicants.
• To test the hypothesis that applicant rankings would significantly change without the Step 1 score.

Main Methods:

• A retrospective analysis of 653 General Surgery residency applications from 2017-2020.
• Development of a scoring system that included USMLE Step 1 and 2, AOA status, grades, clerkship scores, and extracurriculars.
• Comparison of applicant rank lists and score distributions with and without the inclusion of USMLE Step 1 points.

Main Results:

• Removing USMLE Step 1 scores altered the rank of 40% of applicants, with significant shifts between performance tiers.
• The top third of applicants saw 18.8% drop to the middle third, and 11.7% of the bottom third rose to the middle.
• USMLE Step 1 scores created a wider, negatively skewed score distribution; its removal resulted in a narrower, more centered distribution.

Conclusions:

• The USMLE Step 1 score exerted a substantial influence on applicant evaluation and ranking.
• Eliminating the Step 1 score led to a more compressed and potentially fairer ranking of applicants.
• Residency programs can now prioritize other variables that may better align with program-specific core values and applicant potential.