PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Machine Learning Security

Background:

• Pre-trained language models (PLMs) demonstrate remarkable performance but are vulnerable to backdoors, where specific trigger tokens manipulate model behavior.
• Few-shot fine-tuning and prompting are popular NLP training paradigms due to PLM generalizability and high training costs.
• Existing backdoor mitigation methods often require trigger inversion and model retraining, which can be inefficient.

Purpose of the Study:

• To introduce PromptFix, a novel backdoor mitigation strategy for NLP models.
• To address the vulnerability of few-shot fine-tuning and prompting paradigms to backdoor attacks.
• To develop a method that preserves model parameters while effectively neutralizing backdoor triggers.

Main Methods:

• PromptFix employs adversarial prompt-tuning using two sets of soft tokens: one to approximate the trigger and another to counteract it.
• The method avoids explicit trigger inversion and model fine-tuning, keeping original model parameters intact.
• Adversarial optimization is utilized to adaptively balance trigger identification and performance preservation.

Main Results:

• Experiments demonstrate PromptFix's effectiveness against various backdoor attacks in NLP models.
• The method shows strong performance even under domain shift, indicating applicability to models with unknown pre-training data.
• PromptFix successfully neutralizes backdoor triggers without compromising the model's general performance.

Conclusions:

• PromptFix provides an effective and parameter-efficient solution for mitigating backdoors in NLP models within few-shot settings.
• The technique is robust to domain shifts, making it suitable for real-world prompt-tuning applications.
• This adversarial prompt-tuning approach offers a promising direction for enhancing the security of pre-trained language models.