Faculty Narrative Feedback in the Vascular Surgery Entrustable Professional Activity (EPA) Pilot is...

Performance feedback is critical to guide vascular surgery trainees' skill development across competency domains. Workplace-based assessment of Entrustable Professional Activities (EPA) allows faculty frequent opportunities to provide and document clinical performance feedback. ACGME Milestones were previously mapped to each EPA and accompanying behavioral anchors reflect the range of competencies required to execute each clinical activity. It is unclear how faculty use this information to guide and focus the content of their feedback. We sought to identify the core competencies reflected in narrative feedback in a multi-institutional study of EPA assessments.

EPA assessments were collected during a 3-month national Vascular Surgery EPA Pilot in 2024. Trainees were rated on their performance of EPAs on a 4-point Likert scale and faculty had the opportunity to provide feedback. Narrative feedback was qualitatively analyzed using focused coding guided by the Vascular Surgery Milestones 2.0 framework. Feedback was coded using the six ACGME Core Competencies-patient care (PC), medical knowledge (MK), professionalism (PROF), interpersonal and communication skills (ICS), practice-based learning and improvement (PBLI), and systems-based practice (SBP)-to characterize the domains emphasized in feedback. Quality assessment scores were assigned to all narrative feedback using the Quality of Assessment of Learning (QuAL) framework. Descriptive analysis was completed using R.

Faculty completed 983 EPA assessments of 78 trainees across 17 institutions (n=16 integrated residencies, n=12 fellowships). Narrative feedback was included in 760 (77%) assessments. Feedback most frequently addressed PC (n=662, 82%), followed by MK (n=418, 55%), while feedback about PBLI (n=42, 6%), ICS (n =42,6%), PROF (n=20, 3%), and SBP (n=26, 3%) were less common. Most assessments were completed for PGY-3 (29%) and PGY-4 trainees (22%). Most assessments occurred during the intra-operative phase of care (88%), with relatively few occurring in pre-operative (7%) or post-operative (5%) phases. Additionally, feedback referencing PC was associated with higher odds of high-quality feedback in comparison to feedback about the other competencies (OR = 25.56, 95% CI: 6.13-106.55).

In this multi-institutional study of vascular surgery EPA assessments, faculty most frequently provided trainees with feedback on their performance of PC and MK competencies. Feedback on performance related to SBP was infrequently provided and, when provided, was of generally low quality. This has important implications for the use of EPAs to inform ACGME Milestones ratings and suggests alternative assessment tools will continue to be needed to capture performance across all competency domains required of a physician.