Exploring Program Differences in ACGME Milestone 2.0 Ratings in General Surgery: Patient Care and Systems-Based...

Area of Science:

• Medical Education Research
• Surgical Training Assessment

Background:

• The Accreditation Council for Graduate Medical Education (ACGME) Milestones 2.0 are crucial for assessing resident competency.
• Understanding sources of variation in these ratings is essential for accurate evaluation.

Purpose of the Study:

• To quantify program-level contributions to variability in general surgery resident ratings for patient care (PC) and systems-based practice (SBP) Milestones 2.0.

Main Methods:

• Retrospective analysis of national ACGME Milestones 2.0 data (2020-2021) for 7581 general surgery residents across 328 U.S. programs.
• Intraclass correlation coefficients (ICCs) were used to determine the extent of program-level variation in PC and SBP ratings across postgraduate years (PGY1-5).

Main Results:

• Significant program-driven variation was found in all PC and SBP sub-competencies, particularly at PGY1 and PGY5 levels.
• Program factors accounted for 38%-63% of variance in PC ratings and 45%-75% in SBP ratings.
• The highest program influence was in SBP3 (Physician role in health care systems), ranging from 55% to 75%.

Conclusions:

• A substantial portion of variation in PC and SBP Milestone ratings is attributable to program-level factors, not solely resident performance.
• Differences in clinical competency committee practices, institutional culture, and training environments likely drive this variation.
• Further research is needed to identify specific institutional practices influencing Milestone ratings.