Related Experiment Video
Updated: May 31, 2025
06:18
Author Spotlight: Segmentation and VR for Advanced Neurovascular Interventions
Published on: April 5, 2024
951
SAF-IS: A spatial annotation free framework for instance segmentation of surgical tools
Luca Sestini1, Benoit Rosa2, Elena De Momi3
1ICube, University of Strasbourg, CNRS, France; Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milano, Italy.
Medical Image Analysis
|January 24, 2025
Summary
This study introduces a new framework for surgical instrument instance segmentation that avoids costly pixel-level annotations. It uses binary masks and tool presence labels for training, achieving state-of-the-art results without spatial data.
Area of Science:
- Computer-assisted surgery
- Medical image analysis
- Deep learning for surgical robotics
Background:
- Instance segmentation of surgical instruments is critical for computer-assisted surgery applications.
- Current methods rely on fully-supervised deep learning models requiring expensive pixel-level annotations.
- This annotation bottleneck hinders the widespread adoption of advanced surgical tools.
Purpose of the Study:
- To develop a novel framework for surgical instrument instance segmentation that eliminates the need for spatial annotations during training.
- To leverage readily available data like binary tool masks and tool presence labels for effective model training.
- To enable more accessible and scalable development of computer-assisted surgery systems.
Main Methods:
- A framework is proposed that utilizes binary tool masks and tool presence labels for training.
- The method learns to extract individual tool instances and encode them into compact vector representations.
- A small subset of instances is selected for human operator labeling, guiding the training of a tool instance classifier.
Main Results:
- The framework was validated on the EndoVis 2017 and 2018 segmentation datasets.
- Results were demonstrated using both manually annotated and unsupervised predicted binary masks.
- The approach using unsupervised binary masks achieved instance segmentation completely free from spatial annotations.
- Performance surpassed several state-of-the-art fully-supervised segmentation methods.
Conclusions:
- The developed framework offers an effective solution for surgical instrument instance segmentation without requiring expensive pixel-level annotations.
- Utilizing unsupervised binary segmentation models for mask generation results in a fully annotation-free approach.
- This method significantly advances the feasibility of developing advanced computer-assisted surgery applications by reducing data acquisition costs.
