Unsupervised Class Generation to Expand Semantic Segmentation Datasets

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Semantic segmentation requires extensive pixel-level labeling, making it costly and time-consuming.
• Synthetic data and domain adaptation are used to reduce labeling costs, but struggle with novel classes.
• Generative models, especially diffusion models, create high-quality images from text prompts without supervision.

Purpose of the Study:

• To develop an unsupervised pipeline for generating synthetic data with segmentation masks for novel classes.
• To integrate these generated examples into existing semantic segmentation datasets.
• To improve unsupervised domain adaptation by incorporating new classes without altering core algorithms.

Main Methods:

• Leveraging Stable Diffusion for image generation based on text prompts.
• Utilizing the Segment Anything Module for automatic mask generation.
• Developing a method to integrate generated cutouts of novel classes into training datasets.

Main Results:

• Successfully generated class examples with associated segmentation masks.
• Integrated novel class data into semantic segmentation datasets with minimal user input.
• Achieved an average Intersection over Union (IoU) of 51% for novel classes.
• Reduced errors for existing classes, leading to improved overall performance.

Conclusions:

• The proposed unsupervised pipeline effectively generates and integrates novel classes for semantic segmentation.
• This approach enhances unsupervised domain adaptation by expanding training data diversity.
• The method shows significant potential for improving semantic segmentation models without extensive manual annotation or algorithm modification.