Development and validation of Simulation Scenario Quality Instrument (SSQI)

Due to the unmet need for valid instruments that evaluate critical components of simulation scenarios, this research aimed to develop and validate an instrument that measures the quality of healthcare simulation scenarios.

A sequential transformative mixed-method research design was used to conduct the study. The development and validation of the instrument involved two phases: the qualitative phase, which included defining the instrument's theoretical background and instrument construction, followed by the quantitative phase, where the instrument was piloted and validated. The qualitative study included 17 healthcare simulation experts, where three focus group was conducted, and the first version of the instrument was constructed based on the focus group analysis and the theoretical framework constructed using the literature review. During the quantitative phase, the instrument's quantitative piloting included 125 healthcare simulation scenarios; then, the instrument went through construct validity and reliability testing.

Content experts confirmed the theoretical model and instrument framework. The average item content validity index (I-CVI) scores and the average of the I-CVI scores (S-CVI/Ave) for all items on the scale or the average proportion relevance judged by all experts was 0.87. The conformity factor analysis results showed a good fit for the proposed 10-factor model (CFI (the comparative fit index) = 0.998, Tucker-Lewis index = 0.998, Root mean square error of approximation (RMSEA) = 0.061. The final instrument included ten domains: 1. Learning objectives, 2. Target group, 3. Culture, 4. Scenario case, 5. Scenario narrative briefing, 6. Scenario complexity, 7. Scenario flow, 8. Fidelity, 9. Debriefing, and 10.

The SSQI included 44 items that are rated on a 3-point scale (Meets Expectations = (2), Needs Improvement, (1), Inadequate (0)).

This validated and reliable instrument will be helpful to healthcare educators and simulation experts who want to develop simulation-based training scenarios and ensure the quality of written scenarios.