Investigating the Sim-to-Real Generalizability of Deep Learning Object Detection Models

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• State-of-the-art object detection models require extensive datasets for training.
• Training deep learning models on simulation data often leads to a performance drop when applied to real-world images.
• Existing research primarily focuses on data and domain adaptation, neglecting the model's intrinsic properties.

Purpose of the Study:

• To investigate the influence of object detection models themselves on the performance gap between simulation and real-world data.
• To define and evaluate a metric for sim-to-real generalizability.
• To identify which model components contribute most to generalization capabilities.

Main Methods:

• Defined a novel metric: sim-to-real generalizability.
• Trained 12 different deep learning-based object detection models.
• Evaluated 144 model variations by adjusting hyperparameters.

Main Results:

• Demonstrated a clear influence of the feature extractor on sim-to-real generalizability.
• Identified specific correlations and insights into model behavior.
• Showcased significant performance variations across different models and configurations.

Conclusions:

• The choice of feature extractor is critical for the sim-to-real generalizability of object detection models.
• Future research should focus on developing feature extractors with enhanced generalization capabilities.
• Understanding model-specific influences is key to bridging the simulation-to-real gap in AI.