Multi-Institutional Analysis of Entrust Performance With Competency-Based Surgical Education Metrics

Area of Science:

• Medical Education
• Surgical Training
• Assessment Tools

Background:

• The American Board of Surgery (ABS) is transitioning to competency-based medical education, necessitating objective assessment tools.
• The ENTRUST platform, a virtual-patient simulation, has prior validity evidence for assessing Entrustable Professional Activities (EPAs) at a single institution.

Purpose of the Study:

• To gather further validity evidence and assess the generalizability of the ENTRUST platform for evaluating clinical decision-making.
• To investigate ENTRUST's utility as an objective assessment tool in surgical residency.

Main Methods:

• An ENTRUST EPA Assessment was developed with 8 case scenarios (4 Inguinal Hernia, 4 Right Lower Quadrant Pain/Appendicitis).
• 49 general surgery residents (PGY 1-5) from two institutions completed the assessment.
• Bivariate analysis examined correlations between ENTRUST performance and PGY-level, ABS In-Service Training Examination (ABSITE), ACGME Surgery Milestones, and ACGME Case Logs.

Main Results:

• ENTRUST scores significantly correlated with PGY-level (rho=0.58), ABSITE Percent Correct (rho=0.68), and combined Appendectomy/Inguinal Hernia case log volume (rho=0.68).
• Performance on ENTRUST also showed significant correlations with all 18 ACGME Surgery Milestone Levels (rho=0.43 to 0.57), particularly with Patient Care Milestones (PC1-PC4).

Conclusions:

• The ENTRUST platform's performance is significantly correlated with established metrics in surgical education.
• This multi-institutional study strengthens the validity evidence and generalizability of the ENTRUST Assessment Platform.
• ENTRUST serves as an evidence-based tool to objectively measure clinical decision-making, supporting competency-based surgical education.