Faculty evaluation of simulation-based modules for assessment of intraoperative decision making

Previous studies using simulation-based curricula have focused largely on technical skills. We developed a set of simulation-based modules that focus on intraoperative decision making. The objective of this study was to conduct a faculty evaluation of: (1) the usefulness of 4 newly developed, simulation-based modules; (2) the curricular need to train and assess intraoperative decision making skills of the residents; and (3) potential for resident benefit.

Simulation-based modules were developed using a cognitive task analysis (CTA) framework. The CTA framework involved faculty interviews focusing on 4 operative tasks that span a range of complexity: (1) creation of small and large bowel stoma, (2) laparoscopic ventral hernia repair, (3) pancreaticojejunostomy, and (4) lymph node biopsy during a mediastinoscopy. An experienced psychologist conducted task-specific, one-on-one interviews with fellowship-trained specialists who perform these operations in their practice. Two faculty were interviewed for each procedure. The interviews lasted a minimum of 1 hour and focused on critical decisions, error prevention, error recognition, and error rescue strategies. The coded interview summaries were used as development guides for the simulation-based learning modules. Each module included locally developed physical models for the simulated operative tasks combined with oral and paper-based questions. The physical models were fabricated in such a way that simulated operative tasks could be performed using standard surgical instruments. To assess the newly developed simulation-based modules, 8 volunteer faculty (50% overlap with the interview pool) participated in a simulation-based exercise during a one-on-one session and then completed an 8-item survey cast on a 5-point Likert agreement scale (1 = strongly disagree, 5 = strongly agree). One of the items was worded negatively to ensure internal consistency. An independent observer recorded faculty session times and assessed faculty engagement in the task (1 = not engaged, 5 = extremely engaged).

On average, faculty spent 60 minutes completing each simulation-based exercise. Over 80% of this time was spent performing the operative tasks as they would during a real-life procedure. Mean engagement rating was 4.9 (maximum 5.0, SD = 0.3). Survey results show strong agreement on the importance of training and assessing intraoperative decision making, and that residents would likely benefit from the simulation-based modules.

We developed 4 high-fidelity simulation-based modules to assess intraoperative decision making. Faculty agree strongly on the importance and need for additional modules.