On the Adversarial Transferability of Generalized "Skip Connections".

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning Security

Background:

• Skip connections are crucial for deep learning model performance in standard conditions.
• However, their role in adversarial scenarios remains underexplored.

Purpose of the Study:

• To investigate the impact of skip connections on adversarial example transferability.
• To propose a novel method, Skip Gradient Method (SGM), to exploit this property.

Main Methods:

• Analyzing gradient flow in ResNet-like architectures under adversarial attacks.
• Developing SGM to bias backpropagation towards skip connection gradients.
• Extending SGM to diverse architectures like Vision Transformers (ViTs) and Natural Language Processing (NLP) models.

Main Results:

• SGM significantly improves the transferability of adversarial examples across various model families (ResNets, Transformers, LLMs).
• The method remains effective against ensemble attacks, targeted attacks, and models with defenses.
• Empirical evidence and theoretical explanations support SGM's efficacy.

Conclusions:

• Skip connections present a vulnerability exploitable for generating highly transferable adversarial attacks.
• SGM offers a potent technique for adversarial attacks, highlighting architectural vulnerabilities.
• Findings prompt research into secure model architecture design.