Automated task-level autonomy assessment in robotic surgery

Surgical autonomy for trainees has remained elusive to quantify. Proportion of active control time (ACT) of a trainee during a robotic case can be used as a broad measure of autonomy. However, this metric lacks in the granular detail of quantifying at what specific steps trainees were actively participating. We aim to quantify trainee involvement during robotic-assisted hiatal hernia repair at a task-specific level.

We performed a retrospective review of surgical performance data from robotic-assisted hiatal hernia repairs performed. These cases were segmented into 5 tasks by AI-assisted annotation with human review. The segmented tasks included: hiatal dissection, gastric fundus mobilization, mediastinal dissection, cruroplasty and fundoplication. Tasks were excluded if video segmentation of tasks was incorrect. During each task, ACT was recorded for resident, fellow and attending. Resident and fellow ACT per task was compared using the Mann-Whitney U test.

Residents had the highest %ACT in the hiatal dissection (53%), gastric fundus mobilization (84%) and fundoplication (57%) tasks. Fellows had greater than 80% ACT in all 5 tasks, with the highest %ACT in the gastric fundus mobilization (100%) and hiatal dissection (88%). There was a significant difference between resident and fellow ACT during mediastinal dissection and cruroplasty.

This study demonstrates how objective performance metrics and automated case segmentation can quantify trainee participation at a task-specific level. By utilizing data afforded by a robotic surgery platform, we are able to provide an objective and automated form of assessment with minimal impact on the workflow of attendings and residents. Our findings can serve to inform residents on what steps they can expect to be involved in during the procedure, appropriate to their PGY level. With this task-level data, we can provide a roadmap for trainee progression to achieve full surgical autonomy.