Multi-Institutional Analysis of Entrust Performance With Competency-Based Surgical Education Metrics

As the American Board of Surgery (ABS) implements Entrustable Professional Activities (EPAs) and shifts to a competency-based medical education paradigm, there is ongoing need for objective, evidence-based assessment tools to augment microassessments and inform readiness for entrustment. ENTRUST is an online virtual-patient simulation platform with prior validity evidence for its use in assessing EPAs at a single institution. In this multi-institutional study, we aim to collect further validity evidence and generalizability for ENTRUST as an assessment of clinical decision-making.

In alignment with ABS General Surgery EPA Descriptions, an ENTRUST EPA Assessment comprised of 4 Inguinal Hernia case scenarios and 4 Right Lower Quadrant (RLQ) Pain/Appendicitis case scenarios was iteratively developed and scored. Bivariate analysis was performed to examine the relationship of ENTRUST performance to PGY-level, ABS In-Service Training Examination (ABSITE), ACGME Surgery Milestones, and ACGME Case Logs using Spearman's rank and Pearson correlations.

Two geographically diverse academic general surgery residency programs.

PGY 1-5 general surgery residents (n = 49) completed the assessment in proctored examination settings at their respective institutions.

ENTRUST EPA Assessment Total Score was significantly correlated with PGY-level (rho = 0.58, p < 0.001), ABSITE Percent Correct (rho = 0.68, p < 0.001), and ACGME Appendectomy/Inguinal Hernia combined case log volume (rho = 0.68, p < 0.001). ENTRUST performance was also significantly correlated with all 18 ACGME Surgery Milestone Levels (rho = 0.43 to rho = 0.57, all p < 0.005), including strong correlations with Patient Care Milestones (PC1-PC4) (PC-1, rho = 0.54; PC-2, rho = 0.55, PC-3, rho = 0.57; PC-4, rho = 0.54, all p < 0.001).

Performance on ENTRUST was significantly correlated with established metrics in surgical education, including ABSITE, ACGME Case Logs, and ACGME Surgery Milestone Levels. This study further strengthens validity evidence for the ENTRUST Assessment Platform and adds to its generalizability across residency programs. ENTRUST is an evidence-based assessment tool to augment objective measurement of clinical decision-making and support competency-based surgical education.