Endoscope Automation Framework with Hierarchical Control and Interactive Perception for Multi-Tool Tracking in...

Area of Science:

• Robotics and Automation in Surgery
• Computer Vision for Medical Applications

Background:

• Minimally invasive surgery (MIS) heavily relies on visual feedback for surgical tasks.
• Automating endoscopic control is challenging due to multi-agent dynamics and real-time adaptation needs.

Purpose of the Study:

• To introduce a novel framework for adaptive visual field control and robust tool tracking in MIS.
• To ensure optimal viewpoints for surgeons and assistants during surgical procedures.

Main Methods:

• Integration of a Hierarchical Quadratic Programming controller with an advanced interactive perception module.
• Development of a system capable of handling multiple objectives within predefined thresholds for efficient tracking.
• Implementation of adaptive control for varying operating conditions like lighting and occlusions.

Main Results:

• Empirical validation demonstrated robustness and adaptability in single, double, and quadruple tool tracking scenarios.
• The framework successfully maintained efficient tracking during tissue resection tasks.
• System performance was evaluated across diverse and challenging operating conditions.

Conclusions:

• The novel framework shows significant potential for real-world application in minimally invasive surgery.
• Positive user study feedback indicated reduced cognitive and physical strain for surgical teams.
• The system offers improved visual feedback and tool tracking for enhanced surgical outcomes.