Simulation-based assessment of anesthesiology residents' competence: development and implementation of the Canadian...

The specialty of anesthesiology will soon adopt the Competence By Design (CBD) approach to residency education developed by the Royal College of Physicians and Surgeons of Canada (RCPSC). A foundational component of CBD is frequent and contextualized assessment of trainees. In 2013, the RCPSC Anesthesiology Specialty Committee assembled a group of simulation educators, representing each of the 17 Canadian anesthesiology residency programs, to form the Canadian National Anesthesiology Simulation Curriculum (CanNASC) Task Force. The goals were to develop, implement, and evaluate a set of consensus-driven standardized mannequin-based simulation scenarios that every trainee must complete satisfactorily prior to completion of anesthesiology residency and certification. Curriculum development followed Kern's principles and was accomplished via monthly teleconferences and annual face-to-face meetings. The development and implementation processes included the following key elements: 1) Curriculum needs assessment: 368 of 958 invitees (38.4%) responded to a national survey resulting in 64 suggested scenario topics. Use of a modified Delphi technique resulted in seven important and technically feasible scenarios. 2) Scenario development: All scenarios have learning objectives from the National Curriculum for Canadian Anesthesiology Residency. Standardized scenario templates were created, and the content was refined and piloted. 3) Assessment: A validated Global Rating Scale (GRS) is the primary assessment tool, informed by using scenario-specific checklists (created via a modified Delphi technique) and the Anesthesia Non-Technical Skills GRS. 4) Implementation: Standardized implementation guidelines, pre-brief/debrief documents, and rater training videos, guide, and commentary were generated. National implementation of the scenarios and program evaluation is currently underway. It is highly feasible to achieve specialty-based consensus on the elements of a national simulation-based curriculum. Our process could be adapted by any specialty interested in implementing a simulation-based curriculum incorporating competency-based assessment on a national scale.