STF: A Unified Framework for Joint Pixel-Level Segmentation and Tracking of Tissues in Endoscopic Surgery

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Technology

Background:

• Precise tissue segmentation is crucial in endoscopic surgery to prevent complications.
• Current video segmentation methods struggle with long-term robustness due to target loss and challenging conditions like occlusion and motion blur.

Purpose of the Study:

• To develop a robust and accurate automated system for long-term surgical video segmentation and tracking.
• To overcome the limitations of existing methods in maintaining target localization during prolonged endoscopic procedures.

Main Methods:

• Proposed a Unified Framework for Joint Pixel-Level Segmentation and Tracking (STF).
• Integrated a deep learning segmentation network with a Hough Voting Network for tracking.
• Implemented a Bayesian refinement module and an adaptive re-detection mechanism triggered by a tracking evaluation module.

Main Results:

• The STF framework demonstrated superior accuracy and temporal consistency compared to existing segmentation networks in long-term surgical video segmentation.
• Achieved enhanced performance under challenging conditions such as occlusion and motion blur.
• Significantly improved robustness and re-detection capabilities for sustained tissue analysis.

Conclusions:

• The STF framework offers a robust and automated solution for long-term surgical video segmentation and tracking.
• This methodology reduces the dependency on manual intervention, enhancing safety and efficiency in endoscopic minimally invasive surgery.