Development of simulation scenarios for surgeons' non-technical skills evaluation

Area of Science:

• Medical Education
• Surgical Training
• Human Factors in Medicine

Background:

• Non-technical skills (NTS) are crucial for surgical performance and patient safety.
• Accurate assessment of NTS in surgeons is essential for effective training and evaluation.
• Existing methods for measuring NTS may lack specificity and appropriate behavioral anchors.

Purpose of the Study:

• To identify specific simulated scenarios and events for surgeons.
• To discretely and effectively measure non-technical skills (NTS) constructs.
• To appropriately measure NTS using specific behavioral anchors.

Main Methods:

• Experts in NTS accumulated a list of simulated scenarios used for NTS evaluation.
• A survey was used for experts to review scenarios, events, and behavioral anchors.
• Coefficient of variation (CV) was used to assess inter-rater agreement on scenario appropriateness and anchor effectiveness.

Main Results:

• Moderate inter-rater agreement was observed, with 86% of event ratings showing minor discordance.
• The highest-rated events for appropriateness and anchor effectiveness focused on surgeons' leadership and communication.
• Three simulated events were identified as most effective for isolating and evaluating specific NTS.

Conclusions:

• Three specific simulated events were identified to discretely and effectively evaluate surgeons' non-technical skills.
• These validated scenarios can be utilized in the future for robust NTS assessment in surgical training.
• The findings contribute to the development of more precise tools for surgical competency evaluation.