ST-ITEF: Spatio-Temporal Intraoperative Task Estimating Framework to recognize surgical phase and predict instrument...

Area of Science:

• Computer Vision
• Surgical Robotics
• Medical Image Analysis

Background:

• Computer-assisted guidance in surgical robotics can improve accuracy and autonomy.
• Accurate intraoperative workflow estimation is crucial but challenging.
• Current methods lack robust multi-object tracking and phase recognition for complex surgeries.

Purpose of the Study:

• To develop a novel framework for computer-assisted cognition guidance in surgical robotics.
• To enable accurate segmentation and prediction of surgical manipulation using computer vision.
• To quantify intraoperative workflow and recognize surgical phases in keratoplasty.

Main Methods:

• A three-stage Spatio-Temporal Intraoperative Task Estimating Framework was proposed.
• Multiple-object segmentation and feature extraction were combined for surgical manipulation determination.
• Multi-object tracking of surgical instruments and corneas, along with geometric property extraction, was employed.

Main Results:

• The framework achieved competitive class-IoU in segmentation using optimized DeepLabV3 with image filtration.
• Mean phase Jaccard reached 55.58% for surgical phase recognition.
• The framework demonstrated accurate segmentation and phase recognition even under complex disturbances.

Conclusions:

• The proposed framework accurately segments surgical instruments and recognizes phases in keratoplasty.
• This approach has high potential for guiding surgical robots in clinical practice.
• The developed system enhances both operation accuracy and autonomy in robotic surgery.