Are transformer-based models more robust than CNN-based models?

Area of Science:

• Artificial Intelligence
• Deep Learning
• Computer Vision

Background:

• The increasing deployment of AI models necessitates robust performance in open environments.
• Evaluating the robustness of deep learning models, especially transformers and CNNs, is critical.
• Existing robustness metrics may not fully capture real-world performance.

Purpose of the Study:

• To compare the robustness of transformer-based and CNN-based deep learning models.
• To identify the sources of robustness from structural and process perspectives.
• To develop improved evaluation metrics and enhancement methods for AI model robustness.

Main Methods:

• Comparative analysis of transformer and CNN models on robustness metrics.
• Investigation of robustness through Fourier transform and game interaction analysis.
• Development of a calibrated evaluation metric and a blur-based enhancement method.

Main Results:

• Transformer models generally exhibit better robustness than CNN models across various metrics.
• Analysis revealed insights into the underlying mechanisms of transformer robustness.
• The proposed calibrated metric and blur-based method achieved state-of-the-art results.

Conclusions:

• Transformer models offer enhanced robustness for AI deployment.
• New evaluation and enhancement strategies can significantly improve model resilience.
• Achieved state-of-the-art performance on benchmark datasets (CIFAR-10-C, CIFAR-100-C, TinyImageNet-C).