Development and Validation of a Model to Predict Milestone Levels Based on Entrustable Professional Activity...

Area of Science:

• Medical Education
• Graduate Medical Training Assessment

Background:

• Residency programs increasingly require assessment using Accreditation Council for Graduate Medical Education (ACGME) milestones and specialty-defined entrustable professional activities (EPAs).
• Accurate assessment is crucial for resident development and program evaluation.

Purpose of the Study:

• To develop and validate a predictive model for individual resident milestone levels.
• To utilize entrustment-supervision levels of EPAs as predictors for ACGME milestone achievement.

Main Methods:

• A prospective cohort study involving 48 U.S. pediatric residency programs over 3 academic years (2021-2024).
• Multilevel structural equation models were fitted using biannual EPA entrustment-supervision and ACGME milestone data from 4,328 residents.
• Two models were developed: one using 17 EPAs and another using 12 EPAs.

Main Results:

• The models demonstrated excellent internal fit (comparative fit indexes > 0.98).
• External prediction of milestone levels for future cycles showed strong correlations (coefficients ranging from 0.68 to 0.72).
• Both 17-EPA and 12-EPA models exhibited similar predictive capabilities.

Conclusions:

• A robust model exists to predict resident milestone levels from EPA entrustment-supervision data.
• This predictive capability supports the meaningful integration of EPAs and milestones in residency program assessments.
• The findings facilitate more effective and data-driven resident evaluation strategies.