Competency based medical education: assessment of procedural and cognitive skills in gastroenterology residency...

Competence by Design (CBD) was implemented in Canada to assess physician trainees' ability to demonstrate competence via entrustable professional activities (EPAs) utilizing the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE); a score originally validated to assess surgical competence. Our study assessed for differences in O-SCORE utilization between non-procedural (cognitive) and procedural EPA assessments in a real-world setting to inform future critical appraisal of CBD.

Adult Gastroenterology subspecialty EPA assessments from 2019 to 2023 at the University of Alberta were reviewed. Evaluator gender, clinical vs. academic practice, advanced training expertise, EPA type, and O-SCOREs were extracted from each EPA assessment. Competence assessments (achieved, neutral, not achieved) were assigned to each EPA assessment based on local competence committee protocols. Chi-squared testing with 95% confidence intervals were calculated.

2660 EPA assessments were included (1385 cognitive and 1275 procedural). Approximately 70% of total EPA assessments denote competence, with approximately 20% neutral, and < 10% indicating "competence not yet achieved". Proportionally higher cognitive EPA assessments denoted competence compared to procedural EPA assessments (75% vs. 63%, p < 0.01). Evaluator characteristics associated with EPA achievement were male gender, community (vs. academic), and hepatology (vs. luminal GI) practice.

We report low rates of the full range of O-SCORE utilization and the majority amount of EPA assessments denoting competence. Further the proportion of EPAs assessed as success varied by evaluator characteristics (gender, specialty, and practice type). This stresses the need for continued cycles of critical appraisal to improve faculty assessment and feedback in residency trainee progress.