Domain Stylization: A Fast Covariance Matching Framework Towards Domain Adaptation

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Synthetic data generation using computer graphics (CG) is crucial for training robotics and autonomous driving models.
• Significant domain gaps between synthetic and real images limit the effectiveness of purely synthetic training.
• Current rendering limitations hinder the direct application of synthetic data in real-world scenarios.

Purpose of the Study:

• To propose a simple and effective image-level domain adaptation framework to close the synthetic-to-real domain gap.
• To avoid complex Generative Adversarial Network (GAN) training by focusing on feature covariance matching.
• To enhance domain adaptation precision through conditional covariance matching with semantic segmentation.

Main Methods:

• Developed a domain adaptation framework based on matching universal feature embeddings' covariance across domains.
• Introduced a conditional covariance matching approach that iteratively estimates semantic regions.
• Conditionally matched class-wise feature covariance based on estimated segmentation regions for precise alignment.

Main Results:

• Achieved state-of-the-art domain adaptation results by mutually refining segmentation estimation and covariance matching.
• Demonstrated superior performance over existing domain adaptation methods in multiple synthetic-to-real settings.
• Significantly reduced Frechet Inception distance between source and target domains, validating effective domain gap bridging.

Conclusions:

• The proposed framework offers a fast, convenient, and effective solution for synthetic-to-real domain adaptation.
• Conditional covariance matching significantly improves the precision of domain alignment compared to universal matching.
• The approach successfully bridges the domain gap, enhancing the utility of synthetic data for real-world applications.