Multilevel effective surgical workflow recognition in robotic left lateral sectionectomy with deep learning:...

Area of Science:

• Robotics
• Computer Science
• Medical Imaging

Background:

• Automated surgical workflow recognition is crucial for computational models in surgery.
• Accurate recognition and segmentation of surgical procedures are key to advancing autonomous robotic surgery.
• This study focuses on robotic left lateral sectionectomy (RLLS) to enhance surgical process understanding.

Purpose of the Study:

• To construct a multigranularity temporal annotation dataset for RLLS.
• To develop a deep learning model for multilevel surgical workflow recognition (steps, tasks, activities).
• To achieve accurate recognition of both overall and effective surgical workflows.

Main Methods:

• A dataset of 45 RLLS videos was created with temporal annotations for all frames.
• Frames were classified as 'effective' or 'under-effective' based on surgical contribution.
• A hybrid deep learning model was employed for workflow recognition at multiple hierarchical levels.
• Multilevel effective surgical workflow recognition was performed after filtering out under-effective frames.

Main Results:

• The dataset contains over 4.3 million annotated RLLS frames, with over 2.4 million classified as effective.
• Overall accuracies for recognizing steps, tasks, activities, and under-effective frames were 0.82, 0.80, 0.79, and 0.85, respectively.
• Effective surgical workflow recognition achieved higher accuracies: 0.96 for steps, 0.88 for tasks, and 0.82 for activities.

Conclusions:

• A comprehensive RLLS dataset with multilevel annotations was established.
• A hybrid deep learning model demonstrated effective surgical workflow recognition capabilities.
• Removing under-effective frames significantly enhanced recognition accuracy, supporting autonomous robotic surgery.