Occlusion-Resilient Instance Segmentation of Surgical Instrument Parts Using YOLO and Generative Adversarial Networks...

Area of Science:

• Medical Robotics
• Computer Vision
• Image Segmentation

Background:

• Accurate surgical instrument segmentation is vital for safe and precise minimally invasive robotic surgery.
• Real-world surgical scenes present challenges like occlusions, overlapping instruments, and visual noise, hindering conventional models.

Purpose of the Study:

• To develop an advanced framework, SurgSeg-GAN, for robust surgical instrument segmentation.
• To improve the accuracy and reliability of segmentation in complex surgical environments.

Main Methods:

• Proposed SurgSeg-GAN, a hybrid instance segmentation framework combining a fine-tuned YOLOv11 model with a Generative Adversarial Network (GAN).
• The GAN component is designed to generate occlusion masks and recover missing instrument features.
• Validated the framework on the EndoVis 2017 and EndoVis 2018 surgical instrument segmentation datasets.

Main Results:

• SurgSeg-GAN achieved a mean Intersection over Union (mIoU) of 77% and a Dice coefficient of 90% on the EndoVis 2017 dataset.
• On the EndoVis 2018 dataset, the framework reached an mIoU of 71% and a Dice coefficient of 87%.
• Outperformed several state-of-the-art instance segmentation methods, demonstrating enhanced robustness and generalizability.

Conclusions:

• SurgSeg-GAN significantly improves surgical instrument segmentation accuracy in challenging conditions.
• The integration of occlusion-aware GANs enables feature recovery for partially visible instruments.
• The framework contributes to safer and more reliable real-time guidance in robotic-assisted surgery.