Development and validation of objective performance metrics for robot-assisted basic surgical skills in a simulated...

The chicken anastomosis task is a widely adopted model for training robotic suturing and knot-tying. Aim of this study was to develop and validate objective performance metrics for this task on the Single-Port (SP) robotic platform and to establish a proficiency benchmark for training and assessment.

Seven experts in robotic surgery training and Proficiency Based Progression (PBP) developed the metrics for the SP chicken anastomosis task. The proposed metrics were refined among a panel of n = 14 participants with significant experience in PBP and/or robotics and/or the SP system, through a modified Delphi process. Construct validity was assessed by comparing recorded performances of n = 12 expert ( ≥100 SP RARPs) and n = 12 novice surgeons, blindly scored by two reviewers. Proficiency benchmarks were established based on performance distribution.

Delphi consensus achieved 100% agreement on the proposed performance metrics (n = 5 steps, n = 20 errors, n = 1 failure to progress and n = 4 critical errors [CE]). Novices were twice more likely to commit a CE (n = 6 vs. n = 3). Among participants who completed the task without CEs, novices required significantly longer completion times (mean 25.3 min, 95%CI 18.0-32.7 vs. 11.2 min, 95%CI 8.5-13.9; p = 0.003) and committed a significantly higher mean number of errors (12.3, 95%CI 8.6-16.1 vs. 8.1, 95%CI 6.6-9.6, p = 0.033). Calculated Inter-Rater Reliability (IRR: agreements/agreements+disagreements) was 0.96. Proficiency benchmarks were defined as: absence of CEs and < 10 errors.

The proposed metrics showed robust construct validity and reliability, and support PBP training and assessment for the SP platform. The proposed benchmark, as well as its clinical impact, requires confirmation and prospective evaluation.