Design and Validation of a Simulation Scenario and Assessment Tool: A Pilot Project

In 2016, the National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA) launched a Continued Professional Certification (CPC) Program to promote lifelong learning and to facilitate ongoing professional competency checks for practicing certified registered nurse anesthesiologists (CRNA). The use of simulation-based assessment is currently being studied by the NBCRNA for potential use in the CPC Program. The aim of the pilot project was to create and validate simulation scenarios and assessment tools for potential incorporation into the CPC Program. Using the modified Delphi method, an expert panel of eight CRNAs evaluated the validity of four simulation scenarios and the content validity and reliability of four scenario-specific assessment tools. Each of the eight individual surveys assessing simulation scenario validity and assessment tool validity reached ≥ 75% agreement among the expert panelists. Overall content validity index values for the four assessment tools ranged from 0.960 to 0.993. There was a direct relationship between panelists' scores and level of performance on all returned assessment tools. The expert panel validated four simulation scenarios and four accompanying assessment tools. All scenarios and assessment tools were determined to have high content validity and reliability. Using these scenarios and assessment tools would provide unique advantages over standardized assessment methods.