Identifying novel indicators of non-technical skills derived from operative video annotation

Cognitive non-technical skills (NTS), including situation awareness and decision-making, are critical determinants of surgical outcomes. Current NTS assessments depend on expert human observation, which is resource-intensive and difficult to scale. To address this, we investigated whether surgical gestures, derived from annotated video of the surgical field, could serve as objective indicators of cognitive NTS.

A data set of 40 open-source laparoscopic appendicectomy videos was annotated for temporal (for example surgical gestures) and spatial (for example coordinates of actions) indicators of surgical NTS. Using the Non-Technical Skills for Surgeons (NOTSS) tool, 12 expert observers independently assessed decision-making (1-4) and situation awareness (1-4). Multivariable linear regression analysed video-derived indicators predictive of NTS.

Across all videos, a total of 10 385 events were annotated, generating 87 374 data points. The mean cognitive NOTSS rating was 5.6 (s.d. 0.8) out of 8, with decision-making and situation awareness highly correlated (r = 0.8, P  <  0.001). The final multivariable model explained 39.6% of the variance in expert cognitive NOTSS ratings, identifying five predictors: dexterity index, events during the final operative phase, coagulating events, dropping actions, and actions targeting the small bowel.

This study provides evidence that non-technical skills can be inferred from silent video of the surgical field alone. These findings lay the foundations for scalable, automated tools to evaluate surgeons' cognitive processes offering new avenues to improve surgical training, performance and outcomes.