Individual and Common Attack: Enhancing Transferability in VLP Models Through Modal Feature Exploitation.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Vision-Language Pretrained (VLP) models excel in multimodal tasks but are vulnerable to adversarial attacks.
• Current attack methods suffer from limited transferability due to overfitting individual modality features.

Purpose of the Study:

• To propose a novel adversarial attack framework, I&CA (Individual & Common feature Attack), to enhance adversarial example transferability in VLP models.
• To improve the reliability of VLP models in safety-critical applications by addressing their vulnerability to adversarial manipulation.

Main Methods:

• I&CA simultaneously considers individual features within each modality and common features across modalities.
• The framework disrupts single-modality learning by inducing divergence among individual features.
• It suppresses common feature expression during cross-modal interactions and employs augmentation strategies for both modalities.

Main Results:

• The I&CA framework demonstrated an average transferability improvement of 6.15% over the state-of-the-art DRA method.
• Significant performance gains were observed in both cross-model and cross-task attack scenarios.
• The proposed method effectively enhances the transferability of adversarial examples against VLP models.

Conclusions:

• I&CA offers a more robust approach to crafting adversarial examples for VLP models by considering both individual and common features.
• The framework's ability to improve transferability addresses key limitations of existing methods.
• This research contributes to understanding and mitigating adversarial vulnerabilities in multimodal AI systems.