Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Semantic segmentation assigns labels to image pixels, crucial for image understanding.
• Training models on synthetic data for real-world semantic segmentation (SRSS) faces challenges due to domain mismatch.
• Deep Convolutional Neural Networks (CNNs) often fail to generalize from synthetic to real-world data.

Purpose of the Study:

• To develop effective Domain Generalization techniques for Synthetic-to-Real Semantic Segmentation (SRSS).
• To mitigate the domain gap between synthetic and real-world images without using target domain data during training.
• To improve the generalization capabilities of CNNs for semantic segmentation tasks.

Main Methods:

• Proposed Global Texture Randomization (GTR) to diversify source image textures, reducing reliance on texture-specific features.
• Introduced Local Texture Randomization (LTR) to address texture variations in specific image regions.
• Implemented a Consistency between GTR and LTR (CGL) regularization to harmonize randomization mechanisms.

Main Results:

• GTR and LTR significantly improved SRSS performance by learning domain-invariant cues.
• The proposed methods demonstrated superior performance compared to state-of-the-art domain generalization techniques.
• Experiments on GTA5, SYNTHIA, Cityscapes, BDDS, and Mapillary datasets validated the effectiveness of the approach.

Conclusions:

• The proposed texture randomization mechanisms (GTR and LTR) offer a powerful approach for domain generalization in SRSS.
• This method effectively addresses the domain shift problem without requiring target domain data.
• The findings suggest a promising direction for training robust semantic segmentation models for real-world applications.