Improving Semantic Segmentation of Urban Scenes for Self-Driving Cars with Synthetic Images

Area of Science:

• Computer Vision
• Artificial Intelligence
• Autonomous Driving Systems

Background:

• Semantic segmentation is crucial for self-driving car perception.
• Deep neural networks (DNNs) achieve state-of-the-art results but require extensive labeled data.
• Synthetic data offers a cost-effective alternative for augmenting training datasets.

Purpose of the Study:

• To enhance semantic segmentation accuracy for urban scenes.
• To investigate the impact of augmenting the Cityscapes dataset with CARLA-generated synthetic images.
• To evaluate the effectiveness of different synthetic data augmentation strategies.

Main Methods:

• Augmenting the Cityscapes dataset with synthetic images from the CARLA simulator.
• Training MobileNetV2 and Xception DNNs on real-world data, synthetic data, and a mixed dataset (CCM).
• Comparing segmentation accuracy across different training configurations.

Main Results:

• Training on the mixed Cityscapes-CARLA (CCM) dataset improved DNN accuracy compared to using only Cityscapes.
• Augmentation with low-photorealism synthetic data (MICC-SRI) did not improve accuracy.
• Segmentation accuracy did not scale proportionally with the amount of synthetic data used.

Conclusions:

• High-resolution synthetic data from CARLA can effectively augment real-world datasets for improved semantic segmentation.
• The quantity of synthetic data is not the sole determinant of performance; quality and domain relevance are critical.
• Further research is needed to optimize synthetic data augmentation strategies for autonomous driving perception systems.