Related Experiment Video
Updated: Sep 20, 2025
04:48
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
3.0K
A parallel network utilizing local features and global representations for segmentation of surgical instruments.
Xinan Sun1,2, Yuelin Zou1,2, Shuxin Wang1,2
1Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, 135 Yaguan Road, Tianjin, 300350, China.
Summary
This study introduces a new Mask R-CNN based method for segmenting surgical instruments in robot-assisted surgery. The approach enhances surgeon awareness by accurately identifying instrument types and parts, improving automation potential.
Area of Science:
- Medical Robotics
- Computer Vision
- Surgical Technology
Background:
- Accurate segmentation of surgical instruments is crucial for enhancing surgeon awareness in robot-assisted minimally invasive surgery.
- Existing methods may lack the ability to effectively capture both local details and global context of instruments.
Purpose of the Study:
- To propose a novel instance segmentation method for surgical instruments using Mask R-CNN.
- To improve the accuracy and generalization ability of surgical instrument segmentation.
Main Methods:
- A new feature extraction backbone combining convolutional neural network (CNN) and Swin-Transformer branches was developed.
- Skip fusions were implemented in the backbone to integrate local and global features effectively.
- The method was evaluated on the MICCAI 2017 EndoVis Challenge dataset.
Main Results:
- The proposed method achieved state-of-the-art performance, with mean Intersection over Union (mIoU) scores of 0.5873 for type segmentation and 0.7408 for part segmentation.
- Ablation studies indicated that the novel backbone improved mIoU by at least 17%.
- The method demonstrated precise segmentation of surgical instrument contours, particularly at the end tips.
Conclusions:
- The novel method effectively extracts both local and global features for improved surgical instrument segmentation accuracy.
- Enhanced segmentation precision contributes to more accurate localization and pose estimation of surgical instruments.
- This advancement supports the further automation of robot-assisted minimally invasive surgery.
