Image synthesis with class-aware semantic diffusion models for surgical scene segmentation

Area of Science:

• Computer Vision
• Medical Imaging
• Artificial Intelligence

Background:

• Surgical scene segmentation is vital for precision but hindered by limited and imbalanced data.
• Existing generative models often produce non-diverse images and miss critical, small tissue classes.

Purpose of the Study:

• To develop a novel class-aware semantic diffusion model (CASDM) for synthesizing realistic surgical images.
• To address data scarcity and imbalance in surgical datasets.
• To improve the quality and relevance of synthesized surgical images, particularly for critical tissue classes.

Main Methods:

• Proposed a class-aware semantic diffusion model (CASDM) using segmentation maps as synthesis conditions.
• Introduced novel class-aware mean squared error and class-aware self-perceptual loss functions to prioritize less visible classes.
• Pioneered the generation of multi-class segmentation maps from text prompts for conditional image synthesis.

Main Results:

• CASDM effectively generates realistic surgical scene images and corresponding segmentation maps.
• The model demonstrates strong effectiveness and generalizability across diverse datasets.
• Synthesized data significantly enhances the training and validation of surgical segmentation models.

Conclusions:

• CASDM offers a powerful solution for data augmentation in surgical scene segmentation.
• The approach improves image quality and prioritizes critical anatomical structures.
• This work advances the field by enabling more robust and precise surgical segmentation.