Enhancing Adversarial Defense via Brain Activity Integration Without Adversarial Examples.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Neuroscience

Background:

• Large-scale vision-language models (e.g., CLIP) are vulnerable to adversarial attacks, where imperceptible image modifications degrade performance.
• Existing defenses often require knowledge of specific attacks and compromise accuracy on clean data.

Purpose of the Study:

• To develop a robust adversarial defense for vision-language models that overcomes the trade-off between robustness and clean accuracy.
• To leverage human brain activity data for a novel defense mechanism against unknown adversarial attacks.

Main Methods:

• Proposed an adversarial defense integrating human brain activity features with augmented image features using an encoder.
• Maximized similarity between encoder-predicted features and original visual features to achieve human-like visual invariance and data diversity.
• Developed a defense strategy independent of specific adversarial attack knowledge.

Main Results:

• The proposed method significantly improved the robustness of the CLIP model against adversarial attacks.
• Accuracy on clean images was maintained, overcoming the typical performance trade-off.
• Demonstrated robustness against unknown adversarial attacks.

Conclusions:

• Human brain activity data can be effectively utilized to enhance adversarial robustness in vision-language models.
• The proposed method offers a promising solution for defending against diverse and unknown adversarial attacks without performance degradation.
• This research highlights a new direction for AI security by incorporating neuro-inspired principles.