Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Area of Science:

• Medical image analysis
• Computer vision in surgery
• Artificial intelligence in healthcare

Background:

• Accurate semantic image segmentation is crucial for intelligent assistance in robot-assisted minimally invasive surgery.
• The dynamic nature of the human body and surgical procedures necessitates robust machine-vision models.
• Acquiring large, diverse training datasets for surgical imaging is often costly and impractical.

Purpose of the Study:

• To develop and evaluate novel generative adversarial network (GAN) methods for synthesizing usable surgical tool images.
• To address the challenge of limited training data for robust medical image segmentation models.
• To enhance the performance of UNet models for tool segmentation in surgical settings.

Main Methods:

• Examined three novel generative adversarial network (GAN) approaches to generate synthetic tool images.
• Utilized surgical background images and a small set of real tool images for synthesis.
• Incorporated style preservation and content loss into a multi-level loss function for realistic texture generation while preserving background details.

Main Results:

• The best GAN method generated realistic tool textures and preserved local background content.
• UNet models trained with synthetically generated images showed significant performance improvements.
• Achieved 35.7% and 30.6% improvement in mean Dice coefficient and Intersection over Union scores, respectively, compared to training with purely real images.

Conclusions:

• The proposed GAN-based method effectively generates synthetic training data for surgical tool segmentation.
• This approach holds promise for leveraging routine screening endoscopy to create preoperative synthetic data.
• Enables improved intraoperative UNet tool segmentation, advancing robot-assisted minimally invasive surgery.