Related Experiment Video
Updated: Jan 17, 2026
Simulator Training for Endovascular Neurosurgery
Published on: May 6, 2020
Development of simulation scenarios for surgeons' non-technical skills evaluation
Nicholas E Anton1, Brittany Anderson-Montoya2, Amy Holmstrom1
1Department of Surgery, Indiana University School of Medicine, Indianapolis, IN 46202 USA.
Purpose:
The purpose of this study was to identify specific simulated scenarios and events for surgeons that discretely and effectively measure non-technical skills (NTS) constructs and appropriately measure NTS using specific behavioral anchors.
Methods:
Over the course of two rounds, experts in NTS accumulated a comprehensive list of simulated scenarios used at our institution to evaluate NTS. Utilizing a survey, experts were able to review scenarios, events, and behavioral anchors to evaluate their agreement with the appropriateness of the events and effectiveness of the anchors. Coefficient of variation (CV) was used to evaluate agreement between raters. The highest rated events were identified.
Results:
The CV between raters was moderate, and further inspection revealed that 86% of event ratings had discordance within ± 1 point. The highest rated events in terms of appropriateness and behavioral anchor effectiveness evaluated surgeons' leadership and communication.
Conclusions:
In this study, experts identified three simulated events that isolate and discretely evaluate specific NTS among surgeons. In the future, these scenarios can be used to effectively evaluate surgeons' NTS.
Supplementary Information:
The online version contains supplementary material available at 10.1007/s44186-025-00390-6.
