Attack as Defense: Proactive Adversarial Multi-Modal Learning to Evade Retrieval.

Area of Science:

• Computer Science
• Information Security
• Artificial Intelligence

Background:

• Growing concerns regarding information security necessitate robust user-sensitive data protection.
• Multi-modal retrieval technologies present emerging threats, increasing vulnerability to data leakage and malicious exploitation.

Purpose of the Study:

• To introduce a novel approach, Proactive Adversarial Multi-modal Learning (PAML), for safeguarding sensitive data against multi-modal retrieval.
• To develop a method that transforms sensitive data into adversarial representations to ensure privacy.

Main Methods:

• PAML analyzes retrieval feedback mechanisms by querying a knowledge-agnostic system.
• A U-Net-based diffusion model generates a semantic perturbation network to subtly alter data semantics.
• Multi-modal retrieved results and noise shift data semantics towards outliers, preventing neighbor retrieval.

Main Results:

• PAML demonstrated superior performance compared to baseline methods in data privacy protection.
• Ablation studies confirmed the effectiveness of individual components within the PAML framework.
• The approach showed applicability across diverse multi-modal retrieval systems.

Conclusions:

• PAML effectively protects sensitive data privacy by creating adversarial counterparts resistant to malicious multi-modal retrieval.
• The method enhances data security without compromising visual realism or outlier generalization.
• The proposed technique offers a viable solution for securing sensitive information in the era of advanced retrieval technologies.