TUNeS: A Temporal U-Net With Self-Attention for Video-Based Surgical Phase Recognition

Area of Science:

• Computer Vision
• Artificial Intelligence
• Surgical Technology

Background:

• Context-aware computer assistance in surgery requires automatic recognition of surgical phases.
• Video analysis for surgical phase recognition faces challenges in feature extraction and temporal modeling.

Purpose of the Study:

• To explore attention mechanisms for improved temporal modeling in surgical phase recognition.
• To propose TUNeS, an efficient temporal model integrating self-attention within a U-Net structure for enhanced surgical phase recognition.

Main Methods:

• Utilized self-attention mechanisms within a convolutional U-Net architecture (TUNeS).
• Trained Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) on extended video segments for long temporal context.
• Evaluated model performance on the Cholec80 dataset.

Main Results:

• Feature extractors trained with longer temporal context generally improved performance across temporal models.
• TUNeS achieved state-of-the-art performance on the Cholec80 dataset using contextualized features.
• Demonstrated the effectiveness of TUNeS in capturing long-range dependencies for surgical phase recognition.

Conclusions:

• Attention mechanisms can be effectively utilized to build accurate and efficient temporal models for surgical phase recognition.
• Automatic surgical phase recognition is crucial for optimizing surgical workflows and enhancing patient care through context-aware assistance.