Exploring Program Differences in ACGME Milestone 2.0 Ratings in General Surgery: Patient Care and Systems-Based...

Our aim was to quantify the extent to which program-level differences contribute to variation in Milestone 2.0 ratings for ACGME Milestones 2.0 patient care (PC) and systems-based practice (SBP) competencies among general surgery residents.

Retrospective study analyzing ACGME Milestones 2.0 ratings data from general surgery residency programs across the United States for the academic year 2020-2021. Intraclass correlation coefficients (ICCs) were used to measure the extent of program-level variation in ratings across postgraduate year (PGY) levels 1-5.

This study utilized a national dataset of deidentified ACGME Milestone ratings for general surgery residents in the United States, obtained under a data use agreement with the Accreditation Council for Graduate Medical Education (ACGME).

The dataset included 7581 general surgery residents (PGY1 through PGY5) from 328 U.S. general surgery programs. All residents were actively enrolled in accredited residency programs, and complete Milestones data for PC and SBP domains was available for inclusion.

Significant program-driven variation was identified across all PC and SBP sub-competencies, most pronounced (i.e., higher ICC value) at the PGY1 and PGY5 levels. ICCs indicated that program-level factors accounted for 38% to 63% of the total variance in residents' PC ratings and 45% to 75% in SBP ratings. The highest program-level influence was observed in SBP3 (Physician role in health care systems), ranging from 55% to 75%. Uniform resident rating occurred at each PGY level and subcompetency.

Our findings suggest that a substantial portion of the variation in PC and SBP Milestone ratings is driven by program-level factors rather than resident performance alone. This variation may result from differences in clinical competency committee rating practices, institutional culture, and clinical training environments. Future research should explore how specific institutional practices residents' impact Milestones ratings.