Assisted phase and step annotation for surgical videos

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Surgical Workflow Analysis

Background:

• Surgical video annotation is crucial for training deep learning algorithms but is knowledge-intensive and time-consuming.
• Existing annotation methods require significant manual effort and expertise.

Purpose of the Study:

• To develop and evaluate a deep learning-based method for pre-annotating surgical video phases and steps.
• To create a user assistance system to streamline the surgical video annotation process.

Main Methods:

• A convolutional neural network was trained to detect errors and infer temporal coherence in predictions.
• The method was implemented in annotation software and tested on cataract surgery videos via a user study.
• Participants' annotation accuracy and completion times were recorded.

Main Results:

• Users employing the assistance system achieved 7% higher accuracy in step annotation.
• The assistance system reduced annotation time by 10 minutes compared to manual annotation.
• User feedback indicated the system did not hinder or complicate the annotation task.

Conclusions:

• The developed assistance system significantly enhances the efficiency and accuracy of surgical video annotation.
• This deep learning-based approach offers a viable solution to the challenges of annotating complex surgical workflows.
• The validated system can aid in the creation of larger, more accurate datasets for surgical AI training.