Assessing Residency Readiness: Patterns of "Not Yet Level 1" Milestone Ratings in Emergency Medicine Interns

To characterize the extent and domains of variability in Emergency Medicine (EM) residency preparedness by evaluating "Not Yet Level 1" (NotYL1) Accreditation Council for Graduate Medical Education (ACGME) Milestones ratings among US MD-graduated interns at mid-year and end-of-year.

This was a national retrospective analysis of ACGME Milestone assessments for 6893 US MD-graduated EM interns from 2021 through 2024. For each of the 22 EM sub-competencies, we calculated the percentages of interns assigned a NotYL1 rating at both the mid-year and end-of-year assessment periods.

Overall, NotYL1 ratings were < 4%. At mid-year, the mean NotYL1 percentage across sub-competencies was 2.5% (range: 1.8%-3.3%), decreasing to 0.4% (range: 0.2%-1.0%) at end-of-year. At mid-year, the highest proportions of NotYL1 ratings were concentrated in all Systems-Based Practice (SBP) sub-competencies-including Patient Safety (SBP1; 3.0%), Quality Improvement (SBP2; 3.2%), System Navigation (SBP3; 3.0%), and the Physician Role in Health Care Systems (SBP4; 3.3%)-as well as Pharmacotherapy (PC5; 3.1%) and Accountability/Conscientiousness (PROF2; 3.2%). By end-of-year, nearly all interns achieved Level 1 or higher, though PROF2 remained the highest outlier at 1.0%.

US MD graduates demonstrate strong overall readiness for EM residency, with most initial gaps resolving by the end of the first year. However, consistent mid-year deficits in systems-based practice and professionalism (accountability/conscientiousness) highlight critical targets for undergraduate medical school accountability, transition-to-residency curricula, and targeted residency onboarding.