Using outcome harvesting: Assessing the efficacy of CBME implementation

Area of Science:

• Medical Education
• Curriculum Development
• Program Evaluation

Background:

• Queen's University transitioned all 29 postgraduate medical education programs to a competency-based medical education (CBME) curriculum in 2017.
• Assessing the efficacy of this large-scale implementation required a robust program evaluation process.

Purpose of the Study:

• To evaluate the efficacy of the competency-based medical education (CBME) curriculum implementation.
• To identify intended and unintended outcomes of the CBME transition using outcome harvesting.
• To understand the factors contributing to the successful implementation of the CBME curriculum.

Main Methods:

• A systematic document analysis approach was employed, categorizing 443 documents across eight implementation areas.
• Thematic coding was used to organize documents and attribute outcomes via the outcome harvesting method.
• Outcomes were validated through interrater reliability and stakeholder substantiation.

Main Results:

• The outcome harvest identified 38 intended or unintended outcomes directly attributable to the CBME implementation.
• Key themes emerging from the harvest included collaboration, community of practice, and stakeholder commitment.

Conclusions:

• Outcome harvesting provided a robust assessment of CBME implementation efficacy.
• The findings offer insights into requirements for successful curricular innovation, including policy development and increased scholarly activity.
• Learner ownership and stakeholder commitment were identified as critical factors in the transition.