TEMSET-24K: Densely Annotated Dataset for Indexing Multipart Endoscopic Videos using Surgical Timeline Segmentation

Area of Science:

• Surgical data science
• Computer vision
• Medical imaging analysis

Background:

• Manual indexing of surgical videos is time-consuming and hinders systematic analysis.
• Automating surgical video analysis using deep learning is promising but limited by a lack of annotated datasets.
• Trans-anal endoscopic microsurgery (TEMS) requires precise procedural understanding.

Purpose of the Study:

• To introduce TEMSET-24K, a novel, large-scale, open-source dataset for TEMS video analysis.
• To provide a benchmark for evaluating deep learning models in surgical video understanding.
• To facilitate the development of automated indexing systems for surgical procedures.

Main Methods:

• Compilation of 24,306 microclips from TEMS surgeries.
• Annotation of clips using a hierarchical taxonomy: "phase, task, and action" triplets.
• Benchmarking deep learning models, including transformer architectures (ConvNeXt, ViT, SWIN V2) with STALNet.

Main Results:

• Deep learning models achieved high accuracy (up to 0.99) and F1 scores (up to 0.99) for phase segmentation.
• STALNet demonstrated consistent performance across various encoders for well-represented surgical phases.
• The dataset facilitates robust evaluation of surgical video analysis techniques.

Conclusions:

• TEMSET-24K is a valuable resource for advancing automated surgical video analysis.
• The dataset supports the development of AI-driven tools for surgical training and performance assessment.
• This work establishes a new benchmark for surgical data science research.