Adversarial Defense without Adversarial Defense: Enhancing Language Model Robustness via Instance-level Principal...

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Machine Learning Security

Background:

• Pre-trained language models (PLMs) demonstrate significant advancements in natural language processing.
• PLMs are susceptible to adversarial attacks, compromising their reliability in real-world scenarios.
• Existing defense mechanisms often involve computationally expensive adversarial training or data augmentation.

Purpose of the Study:

• To develop an efficient add-on module for enhancing the adversarial robustness of PLMs.
• To mitigate adversarial vulnerabilities without relying on traditional defenses or modifying training data.
• To maintain model performance and generalization while improving robustness.

Main Methods:

• Proposes a novel module that removes instance-level principal components from the embedding space.
• Transforms embeddings to approximate Gaussian properties, reducing susceptibility to perturbations.
• Aligns embedding distributions to minimize adversarial noise impact on decision boundaries.

Main Results:

• The proposed method significantly improves adversarial robustness across eight benchmark datasets.
• Maintains comparable accuracy to baseline models before adversarial attacks.
• Demonstrates a balanced trade-off between enhanced robustness and generalization capabilities.

Conclusions:

• The add-on module offers an effective and computationally efficient solution for improving PLM adversarial robustness.
• The approach enhances model security by transforming the embedding space, not by altering training procedures.
• This method provides a practical way to deploy robust PLMs in security-sensitive applications.