Scenario-Driven Synthetic Data Generation Framework for Visual Perception Evaluation Under Adverse Driving Conditions

Developing and evaluating visual perception systems for autonomous vehicles requires data across diverse and adverse driving conditions, yet collecting and annotating such real-world data is costly and often impractical. To address this challenge, we propose a modular, scenario-driven framework for generating synthetic datasets tailored to the evaluation of visual perception functions. The framework aligns with the operational boundaries and detection-response requirements of automated driving functions and comprises three stages: (1) configuring use-case-driven scenarios, (2) generating sensor data and ground truth via simulation, and (3) post-processing to ensure dataset usability. Designed to be generic and flexible, the framework is instantiated and demonstrated through its integration with specific platforms and tools, namely Pro-SiVIC and RTMaps. We evaluate the generated dataset from two perspectives, image fidelity and perception performance under synthetic weather conditions, in comparison to real-world conditions. Furthermore, we train multiple perception models under different learning paradigms, including baseline, transfer-learning, and mixed-training strategies, to examine the influence of synthetic data on robustness. Experimental results demonstrate not only the high quality of the generated data but also its effectiveness in evaluating visual perception functions, as well as its benefit to model robustness and generalization.