A human-AI collaborative framework for surgical field-of-view adjustment: design and experimental validation

The goal of robotic endoscope holders is to replace human assistants by enabling optimal field of view (FoV) adjustment while ensuring stable visualization during minimally invasive surgery (MIS). However, existing systems struggle to achieve this goal, relying either on autonomous strategies that fail to interpret the surgical true intent or on detailed commands that increase the surgeon's interaction burden. Drawing inspiration from clinical decision-making processes, we introduce a novel Human-AI collaborative framework for surgical FoV adjustment (HIC-FoV) to address these limitations. This framework requires only concise voice commands from the surgeon, while leveraging AI to interpret the surgical scene and generate intent-aligned FoV adjustments. At the core of HIC-FoV is a novel multi-dimensional weighting mechanism, which evaluates the surgical scene based on directional and spatial congruence with commands, contextual significance, and perceptual confidence to enable efficient FoV optimization. Experiments conducted on peg transfer and simulated cholecystectomy tasks demonstrated that HIC-FoV achieved performance comparable to manual control (completion time: $127.3\pm 21.5$ s vs $121.4\pm 27.4$ s, $p=0.513$ ) while significantly reducing surgeon workload (NASA-TLX physical demand: $29.3\pm 4.2$ vs $53.5\pm 7.5$ , $p<0.001$ ). Moreover, its stability across varying task complexities and user experience levels allows the robotic endoscope holder to perform on par with an experienced surgical assistant. This work establishes a replicable paradigm for human-AI collaboration in robotic-assisted surgery and holds promising potential for applications in telesurgery.