Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attack

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Vision-language pre-training (VLP) models are powerful but vulnerable to multimodal adversarial examples (AEs).
• Improving the transferability of AEs across different models is crucial for developing robust VLP systems.
• Existing methods for AE generation offer limited improvements in transferability.

Purpose of the Study:

• To develop a novel approach for generating more transferable adversarial examples (AEs) for VLP models.
• To enhance the robustness of VLP models against adversarial attacks.
• To address the limitations of current AE generation techniques.

Main Methods:

• Proposed adversarial evolution triangles, sampling from clean, historical, and current adversarial examples to increase AE diversity.
• Introduced a semantic-aligned subspace to reduce feature redundancy and improve feature matching.
• Generated AEs within a semantic image-text feature contrast space.

Main Results:

• The proposed method significantly improved the transferability of adversarial examples.
• The adversarial evolution triangles and semantic-aligned subspace effectively enhanced adversarial diversity and reduced feature redundancy.
• Experimental results demonstrated superior performance compared to state-of-the-art adversarial attack methods.

Conclusions:

• The novel AE generation strategy effectively enhances adversarial transferability for VLP models.
• Leveraging adversarial evolution triangles and semantic feature spaces offers a promising direction for robust AI development.
• The proposed method provides a practical approach to identifying and mitigating vulnerabilities in VLP models.