Feedback Frequency in Competence by Design: A Quality Improvement Initiative

Area of Science:

• Medical Education
• Otolaryngology-Head and Neck Surgery
• Competency-Based Medical Education

Background:

• Canadian otolaryngology residency programs are adopting Competence by Design (CBD), a competency-based medical education model.
• CBD necessitates frequent, low-stakes assessments and increased feedback, which pilot programs initially struggled to achieve.
• The University of Toronto's program reported only one assessment every four weeks, insufficient for effective CBD implementation.

Purpose of the Study:

• To enhance assessment completion frequency to once per resident per week.
• To utilize quality improvement methodologies to overcome barriers in assessment completion.
• To support the effective implementation of Competence by Design in surgical residency.

Main Methods:

• Engaged residents and faculty to identify barriers to assessment completion.
• Implemented brief electronic assessment forms on mobile devices during face-to-face encounters.
• Tracked assessment completion rates over 10 months and analyzed using statistical process control charts.

Main Results:

• An initial intervention combining a rule set and weekly reminders was ineffective.
• A subsequent audit-and-feedback leaderboard system, displaying weekly resident completion rates, significantly improved performance.
• Assessment completion increased from 0.22 to 2.87 assessments per resident per week over 10 weeks.

Conclusions:

• A resident-designed audit-and-feedback leaderboard system effectively increased the frequency of Competence by Design assessment completion.
• This intervention demonstrates a successful strategy for improving feedback mechanisms in competency-based medical education.
• The findings offer a scalable solution for residency programs implementing CBD.