Preliminary validity evidence for a platform-specific assessment tool for robotic setup and docking

There is currently no widely available, structured assessment tool with reported validity evidence for robotic system setup and docking. A Delphi consensus-derived checklist for the da Vinci Xi platform has been developed previously. The present study aimed to gather preliminary validity evidence for this checklist using Messick's unified framework and Kane's validation inferences. Fifty-one participants without prior robotic experience completed a structured four-day basic robotic skills course. Docking and setup performance was assessed at the start (day 1) and end (day 4) of the course by an independent, ALSGBI-certified robotic trainer using the 69 point checklist. Five dry-lab tasks were assessed at both time points using the Global Evaluative Assessment of Robotic Skills (GEARS) score. Internal structure evidence was obtained from a pre-study calibration exercise and a post-hoc inter-rater reliability analysis in which three independent, blinded raters scored 12 randomly selected procedure videos. Consequences were assessed through Wilcoxon signed-rank comparison of pre- and post-training scores. Relationship to other variables was examined exploratorily comparing checklist and GEARS scores with Spearman correlation coefficient. Inter-rater reliability across 12 videos and three raters demonstrated ICC = 0.91 (95% CI 0.75-0.97) for total scores. Docking scores increased from a median of 43.0 (IQR 34.0-47.5; 62.3%) to 58.0 (IQR 53.0-61.0; 84.1%), a relative increase of approximately 35% (p < 0.001; r = 0.87). A weak but significant positive correlation between docking and composite GEARS scores was observed at the second time point (Spearman ρ = 0.31, 95% CI 0.04-0.54; p = 0.027), with the interrupted suture task showing the strongest individual association (ρ = 0.38, 95% CI 0.12-0.59; p = 0.006). This study provides preliminary validity evidence supporting the use of a Delphi-derived checklist for the structured assessment of robotic setup and docking on the da Vinci Xi platform. Further evidence, particularly from multi-centre studies, more raters, and comparison across experience levels, is required before the tool can be recommended for accreditation purposes.