Image Synthesis Pipeline for CNN-Based Sensing Systems.

Area of Science:

• Computer Vision
• Machine Learning
• Sensor Technology

Background:

• Convolutional Neural Networks (CNN)-based sensors are crucial for applications like medical analytics and autonomous driving.
• The performance of these sensors heavily relies on the quantity and quality of training datasets.
• Existing methods face challenges in generating sufficient, high-quality data.

Purpose of the Study:

• To address data quantity and quality challenges in training datasets for CNN-based sensors.
• To propose and validate a novel approach for improving training datasets.
• To enhance the accuracy and reliability of ML-enabled smart sensor systems.

Main Methods:

• Developed a content creation pipeline using computer graphics and generative neural networks for synthetic data augmentation.
• Generated realistic image sequences with controllable variations in geometry, materials, and lighting.
• Validated the approach on object classification, detection, depth buffer reconstruction, and panoptic segmentation tasks.

Main Results:

• The proposed pipeline generates well-controlled, reproducible datasets with desired feature distributions.
• Synthetic data augmentation significantly improved the accuracy of CNN-based sensors compared to using real-life data alone.
• The method demonstrated effectiveness across diverse computer vision applications.

Conclusions:

• Synthetic data generation is a viable and effective strategy to overcome limitations in real-world datasets.
• The developed pipeline offers a reproducible method for creating high-quality, diverse training data.
• This work contributes to advancing the performance and applicability of ML-enabled smart sensor systems.