Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition.

Area of Science:

• Computer Vision
• Medical Informatics
• Surgical Robotics

Background:

• Automated recognition of surgical activities (steps) is vital for intra-operative computer assistance.
• Current vision-based methods require large amounts of manually annotated data, which is costly and time-consuming to produce.
• Domain-specific expertise is needed for accurate annotation, posing a significant bottleneck.

Purpose of the Study:

• To develop a method for surgical step recognition using coarser, easier-to-annotate phase labels as weak supervision.
• To reduce the dependency on extensively annotated videos for training activity recognition models.
• To enable more efficient development of intelligent surgical assistance tools.

Main Methods:

• Proposed a novel approach using phase-level activity labels as weak supervision for step recognition.
• Introduced a step-phase dependency loss function to leverage the weak supervision signal effectively.
• Employed a Single-Stage Temporal Convolutional Network (SS-TCN) with a ResNet-50 backbone for end-to-end training.
• Utilized weakly annotated videos for temporal activity segmentation and recognition.

Main Results:

• Demonstrated the effectiveness of the proposed weakly supervised method on a large dataset of laparoscopic gastric bypass procedures (40 videos).
• Validated the approach on the public CATARACTS benchmark dataset (50 cataract surgeries).
• Achieved accurate temporal activity segmentation and recognition with reduced annotation effort.

Conclusions:

• Weak supervision using coarser phase labels is a viable and efficient strategy for surgical step recognition.
• The proposed SS-TCN model with step-phase dependency loss effectively learns from weakly annotated data.
• This method significantly lowers the annotation burden, facilitating the development of intelligent intra-operative assistance.