Empirical Evaluation of Invariances in Deep Vision Models

Area of Science:

• Computer Vision
• Deep Learning
• Machine Learning

Background:

• Deep learning models need to be robust to image variations for real-world applications.
• Invariances, like handling transformations, are crucial for reliable computer vision systems.

Purpose of the Study:

• To empirically evaluate modern convolutional neural networks (CNNs) and vision transformers (ViTs) on image invariances.
• To assess model robustness across object localization, recognition, and semantic segmentation tasks.

Main Methods:

• Tested thirty CNN and ViT models on benchmark datasets (COCO, ImageNet).
• Introduced controlled perturbations (blur, noise, rotation, scale) to evaluate robustness.
• Used metrics like mIoU and classification accuracy (Acc) to quantify performance degradation.

Main Results:

• ViTs outperformed CNNs on blur and noise for recognition tasks.
• Both CNNs and ViTs showed vulnerabilities to rotation and extreme scale transformations.
• Semantic segmentation models, particularly SegFormer and Mask2Former, exhibited higher resilience to geometric variations.

Conclusions:

• Current deep learning models have significant vulnerabilities to common image transformations.
• Segmentation models show promise for robustness, but further research is needed for rotation and scale invariance.
• Findings challenge assumptions and offer insights for developing more robust vision systems.