ESIP: Explicit Surgical Instrument Prompting for Surgical Workflow Recognition

Area of Science:

• Computer-assisted surgery
• Medical image analysis
• Surgical robotics

Background:

• Surgical workflow recognition (SWR) is crucial for computer-assisted surgery, aiming to identify phases within surgical videos.
• Current deep learning methods often implicitly extract spatio-temporal features, potentially overlooking critical spatial information like surgical instruments.
• This limitation hinders the accurate identification of surgical phases.

Purpose of the Study:

• To propose an Explicit Surgical Instrument Prompting (ESIP) approach to enhance SWR by explicitly leveraging surgical instrument information.
• To improve the extraction of intra-frame spatial features and inter-frame spatio-temporal features for more accurate phase recognition.
• To develop a single-task SWR framework optimized for feature extraction, distinct from multi-task approaches.

Main Methods:

• ESIP utilizes surgical instrument segmentation to create instrument-specific visual prompts.
• These prompts guide a frozen pre-trained backbone to extract crucial spatial features.
• A SAM-based segmentation with prompt tuning strategy is employed for efficient integration of segmentation features.

Main Results:

• The ESIP method demonstrated superior performance compared to 16 state-of-the-art (SOTA) methods across Cholec80, M2CAI, and AutoLaparo datasets.
• Achieved high Precision (up to 91.8%), Recall (up to 92.2%), and Jaccard index (up to 83.3%).
• Outperformed existing methods in surgical phase recognition tasks.

Conclusions:

• ESIP effectively addresses the limitations of implicit feature extraction in SWR by incorporating explicit instrument information.
• The proposed method offers a significant advancement in computer-assisted surgery through improved surgical workflow recognition.
• The single-task, prompt-guided approach provides a robust framework for future SWR research and applications.